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EXECUTIVE  SUMMARY 

Over  the  last  decade,  the  Edgewood  Chemical  Biological  Center  (ECBC)  conducted  numerous  surveys  of 
technologies.  Until  now,  these  surveys  relied  on  manufacturer  supplied  information  without  a 
laboratory  or  field  assessment  to  corroborate  the  system  performance.  The  Biosensors  Test  Bed  offers  a 
unique  opportunity  to  assess  system  performance  in  an  International  Organization  for  Standardization 
(ISO)  17025-compliant  laboratory  and  in  operationally  relevant  real-world  settings. 

Over  an  18  month  period,  ECBC  scientists  partnered  with  soldiers  to  systematically  down-select  systems 
identified  through  market  surveys  and  open  data  calls.  Beginning  with  40  biological  agent  identifiers,  the 
Test  Bed  staff  chose  the  top  16  candidate  systems  for  secondary  assessment.  The  goal  during  this  down- 
selection  was  to  identify  systems  that  would  represent  a  diverse  collection  of  detection  methodologies, 
with  consideration  of  the  stage  of  development,  assay  availability,  and  logistics.  Inability  to  acquire  a 
working  system  or  commercial  unwillingness  to  participate  reduced  the  number  of  candidate  systems  to 
12  as  the  Test  Bed  moved  into  the  laboratory  assessment.  Overall,  the  laboratory  scientists  and  field 
operators  agreed  that  the  standout  devices  were  the  antibody-based  MesoScale  Diagnostics  Cartridge 
Reader,  the  IQuum  Lab  in  a  Tube  (Liat™)  polymerase  chain  reaction  (PCR)  device,  and  the  BioFire™ 
Diagnostics  FilmArray@  PCR  device.  While  each  of  these  devices  would  benefit  from  more  portability  and 
minor  re-engineering,  user  ratings  indicated  these  devices  were  user-friendly  and  reliable.  On  the  other 
end  of  the  spectrum,  the  Sandia  National  Laboratories'  SpinDx™,  Seattle  Sensors  Systems'  SPIRIT™  and 
the  Research  International  RAPTOR  scored  poorly  and  assessed  to  be  unreliable  and  difficult  to  use. 

The  value  of  laboratory  testing  complemented  by  field  utility  assessments  was  evident  with  these  lower- 
scoring  systems.  Though  sensitivity  and  reliability  issues  were  encountered  in  the  laboratory,  operators 
were  sometimes  able  to  overcome  these  issues  by  working  with  the  company,  rebooting,  or  flushing  the 
devices.  However,  in  the  field  when  the  operators  encountered  system  failures,  fluidic  clogs,  or  software 
errors  they  were  unable  to  rectify  these  issues  and  the  exercise  failed. 

Several  devices  scored  well  because  of  their  small  size  and  simplicity,  but  will  require  re-engineering  to 
increase  applicability  to  military  environments.  About  the  size  of  a  paperback  book  and  operated  with  a 
single  button,  the  Epistem  Genedrive™  PCR  device  is  an  example  of  this  group.  The  device  was  created 
for  a  clinical  diagnostic  market  and  showed  great  potential  as  a  low-cost  small-footprint  highly  sensitive 
biological  identifier  in  the  laboratory;  however,  significant  re-engineering  will  be  required  to  make  this 
device  conducive  to  field  use.  Because  small  size  is  a  top  priority  for  many  military  users,  the  ANP 
Nanolntelligent  Detection  System  (NIDS@)  with  Stand  Alone  Reader  is  a  device  currently  being  used  by 
some  in  the  field.  However,  in  this  assessment,  although  the  NIDS  was  easy  to  use,  it  was  plagued  with 
sensitivity  and  specificity  issues  typical  of  other  hand-held  antibody-based  assay  systems.  On  the  other 
end  of  the  size  spectrum  is  the  Luminex  MAGPIX@,  which  is  slightly  larger  than  a  computer  tower  with  an 
accompanying  laptop.  The  MAGPIX  is  an  antibody-based  system  that  has  high  assay  flexibility  with  the 
potential  for  highly  multiplexed  and  high  throughput  analysis.  In  order  for  the  system  to  be  of  military 
utility  in  the  field,  it  will  require  significant  re-engineering.  As  it  stands  right  now,  the  MAGPIX  is  a 
reliable  laboratory  based  instrument. 

The  Biosensors  Test  Bed  clearly  demonstrated  the  value  of  conducting  both  laboratory  analyses  and  a 
field  based  assessments.  Table  top  assessments  based  on  manufacturer  supplied  data  tell  only  half  the 
story  and  many  devices  that  appeared  favorable  on  paper  were  revealed  to  have  flaws  and  areas 
requiring  restructuring  when  taken  to  the  field.  In  closing,  it  is  important  to  note  that  the  diverse  needs 
of  the  warfighter  mean  that  no  single  device  will  fulfill  all  missions  and  programmatic  requirements. 
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INTRODUCTION 

Biological  detection  systems  have  benefited  from  the  progress  made  in  a 
variety  of  scientific  and  engineering  disciplines.  The  development  and  use  of 
sophisticated,  often  multiplexed,  molecular  biology  assays  along  with 
engineering  developments  such  as  inexpensive  light-emitting  diode  (LED) 
illumination,  charge-coupled  device  (CCD)  imaging,  high-speed  computer 
processing,  and  lightweight  batteries  illustrate  this  technical  convergence. 

With  the  application  of  these  advances,  the  current  generations  of  detection 
systems  are  faster,  have  a  greater  sample  throughput,  and  produce  far  more 
data  than  their  predecessors.  In  addition  to  performance  improvements,  there  is  now  a  greater  diversity 
of  detection  systems  than  ever  before.  Commercially  available  platforms  vary  in  size,  form,  cost,  and 
ease-of-use. 

Despite  this  variety  of  technologies,  there  is  no  ideal  system  that  can  meet  the  wide  range  of  mission 
requirements  and  programmatic  objectives  within  the  federal  government.  It  is  essential  for  program 
managers  and  other  decision  makers  to  explore  the  available  options  prior  to  making  development  and 
acquisition  investment  decisions.  For  biological  detection  systems  investments,  an  analysis  of 
alternatives  should  include  not  only  a  comprehensive  characterization  of  the  available  platforms,  but 
also  3rd  party  validation  of  manufacture  claims.  This  type  of  thorough  evaluation  not  only  serves  to 
match  the  appropriate  system  to  a  program  or  operational  scenario,  but  it  can  identify  platforms  to 
provide  common  solutions  for  similar  federal  interagency  initiatives. 

The  purpose  of  this  study  was  to  provide  an  unbiased  technical  evaluation  of  candidate  biological 
detection  systems  in  support  of  future  Department  of  Defense  (DoD)  and  interagency  acquisition 
decisions.  Because  the  effort  wasn't  tied  to  an  acquisition  program,  this  study  had  the  flexibility  to 
examine  a  wide  variety  of  performance  criteria  to  benefit  diverse  federal  initiatives.  As  a  result,  this 
study  is  unique  for  several  key  reasons.  The  first  distinguishing  feature  is  that  it  evaluated  systems  in 
both  an  ISO  17025  accredited  laboratory  setting  with  highly  skilled  scientists  as  well  as  in  a  field 
environment  with  end-users.  System  performance  can  dramatically  change  under  different  operating 
environments,  while  the  routines  and  habits  of  trained  biologists  in  a  laboratory  setting  compared  to  a 
responder  or  other  end  user  in  a  field  setting  may  also  impact  test  results.  Furthermore,  safety 
equipment,  lack  of  laboratory  infrastructure  and  field  operating  procedures  can  affect  or  even  prevent  a 
system  from  being  used  as  the  manufacture  intended.  A  second  unique  aspect  of  this  study  is  that  it  not 
only  systematically  categorizes  and  ranks  the  attributes  of  biological  detection  systems,  but  it  also 
scores  them  using  an  adjustable  performance  scoring  matrix.  Thus,  the  weights  of  the  scoring  can  be 
adapted  to  fit  different  programmatic  requirements.  The  expectation  is  that  these  amendable  results 
will  reveal  instances  where  the  same  device  may  be  used  for  different  programs  with  overlapping  goals 
and  requirements.  This  may  lead  not  only  to  leveraged  funding,  but  to  a  more  cohesive  strategy  for 
biothreat  detection. 

Importantly,  the  results  of  this  study  favored  smaller  devices  that  are  suited  to,  or  have  the  capability  to 
be  developed  into  a  device  that  may  be  used  in  a  field  environment.  It  is  also  crucial  to  note  that  some 
of  the  tested  systems  are  still  being  developed  and  that  this  study  represents  just  a  snapshot  of  their 
performance.  This  study  could  serve  as  the  basis  for  follow-on  evaluations  as  these  devices  evolve  and 
mature. 
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BACKGROUND 

The  DoD  JPEO-CBD's  JPM-TMT  (now  part  of  JPM-MCS)  conducted  a  technical  information  survey  in  early 
2012  and  subsequent  hands-on  evaluations  of  existing  biological  agent  identifiers.  An  open  Request  for 
Information  (RFI)  was  issued  with  the  intent  of  soliciting  input  from  industry  and  federal  laboratories  to 
identify  existing  equipment  capable  of  field-  and  laboratory-based  pathogen  identification.  The  objective 
of  this  report  is  to  provide  a  snapshot  of  commercially  available  biosensor  systems  that  may  be 
applicable  for  biological  agent  detection  and  identification.  This  report  not  only  provides  a  summary  of 
the  physical  characteristics  of  these  systems,  but  it  details  the  performance  of  these  systems  in  both 
laboratory  and  field  settings. 

In  contrast  to  the  plethora  of  biological  identifier  instruments  available  to  dedicated  stationary 
diagnostic  or  research  laboratories,  the  number  of  such  systems  dedicated,  or  even  applicable,  to  man- 
portable  field  environments  or  mobile  laboratories  is  somewhat  limited.  The  40  responses  to  the  RFI, 
which  included  biological  agent  identifiers  in  stages  of  development  from  concept  to  commercial  off- 
the-shelf  (COTS)  instruments,  were  reviewed  and  initially  sub-divided  into  the  following  categories 
based  on  size  and  detection  technology: 

•  Handheld/Man-portable  -  Antibody-Based 

•  Handheld/Man-portable  -  Nucleic  Acid-based 

•  Mobile  Lab  -  Antibody-based 

•  Mobile  Lab  -  Nucleic  Acid-based 

Information  submitted  in  response  to  the  RFI  was  the  foundation  for  this  subdivision  and  subsequent 
characterization;  however,  other  relevant  sources  of  information  were  also  utilized  (e.g.,  Emanuel,  P.  & 
Caples,  M.  (2011)  The  Chemical,  Biological,  Radiological  Technology  Survey).  Information  gathered  by 
other  ongoing  United  States  Government  (USG)  technology  systems  evaluations,  including  the  Defense 
Threat  Reduction  Agency's  "24  Month  Challenge"  and  the  JPEO-CBD's  Common  Analytical  Laboratory 
System,  were  also  utilized  for  the  initial  assessment  of  biological  identifiers. 

A  down-selection  of  the  instruments  by  ECBC  and  JPM-TMT  staff  generated  a  list  of  16  candidate 
systems  for  a  secondary  assessment.  The  goal  during  this  down-selection  was  to  identify  systems  that 
would  represent  a  diverse  collection  of  detection  methodologies,  with  consideration  given  to  stage  of 
development,  assay  availability,  and  logistics.  Selection  of  nearly  duplicate  systems  was  avoided  so  that 
the  assessments  could  focus  on  distinct  technologies.  After  further  assessment  and  characterization, 
twelve  candidate  systems  were  chosen  for  inclusion  in  the  formal  test  bed  assessment.  Candidate 
technologies  were  evaluated  for  their  ability  to  detect  known  samples  in  an  ISO  17025  compliant 
laboratory  at  ECBC  in  order  to  document  performance  and  verify  vendor  claims.  The  instruments  were 
directly  compared  by  challenging  them  with  identical  preparations  of  inactivated  biological  threat 
agents. 

In  addition  to  the  comprehensive  laboratory  assessment  performed  by  skilled  personnel,  the  systems 
were  evaluated  by  end-users  in  several  operationally  relevant  environments.  Specifically,  the  field 
assessments  were  conducted  by  operators  from  the  Army's  5th  Special  Forces  Group  (Airborne),  the 
Army's  22nd  Chemical  Battalion  and  the  Army's  20th  Support  Command.  The  field  assessments  were 
conducted  at  ECBC's  Skippers  Point,  a  non-laboratory  site  of  former  military  housing  at  Aberdeen 
Proving  Ground  that  is  now  used  to  simulate  operational  situations,  and  within  the  20th  Support 
Command's  Heavy  Mobile  Expeditionary  Laboratory  (HMEL). 
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Figure  1.  Members  of  the  Army  5  Special 
Forces  Group  (Airborne)  perform  sample 
analysis  using  the  BioFire  RAZOR  EX  at  ECBC's 
Skippers  Point  operation  simulation  site. 


i-igure  z.  a  memoer  ot  me  Army  zz  uiemical 
Battalion  (right)  analyzes  a  sample  using  the 
Research  Internal  RAPTOR  under  the  watchful 
eye  of  an  ECBC  scientist  at  Skippers  Point. 


♦ 


Figure  3.  The  Army  20th  Support  Command's 
HMEL  at  Aberdeen  Proving  Ground  served  as 
the  site  for  the  biological  agent  identifiers' 
mobile  laboratory  assessments. 


Figure  4.  Army  20th  Support  Command  HMEL 
interior. 


Performance  data  and  user  input  were  captured  in  data  sheets  and  questionnaires  and  incorporated 
into  multiple  scoring  matrices,  which  allowed  for  the  comparison  of  identification  systems.  For  rapid  and 
easy  reader  assessment,  this  report  presents  a  two-page  summary  of  the  evaluation  of  each  biological 
agent  identifier  system  at  the  beginning  of  the  laboratory  evaluations.  The  reviewers  also  assigned 
Technology  Readiness  Levels  (TRL)  to  each  system,  based  on  DoD  guidance  (Appendix  A).  Acquisition 
managers  will  be  able  to  triage  individual  and  classes  of  detection  systems  with  high-confidence 
performance  data,  weighting  the  data  as  appropriate  to  satisfy  specific  program  requirements.  This 
report  is  expected  to  support  existing  and  future  JPEO-CBD,  DoD,  and  other  USG  acquisition  efforts,  and 
could  be  expanded  to  include  other  surveillance  technologies  as  appropriate.  This  report  complements 
other  reviews  of  contemporary  technologies  to  counter  threats  from  weapons  of  mass  destruction, 
including  The  Chemical  Biological  Radiological  Technology  Survey  (2011;  Emanuel,  P.  &  Caples,  M.);  The 
2013  Nucleic  Acid  Purification  Market  Survey  and  CBRN  Sample  Preparation  Horizon  Scan  (Betters,  J., 
Emanuel,  P.  &  Caples,  M.);  and  The  2013  Global  CBRN  Detector  Market  Survey  and  Horizon  Scan 
(Emanuel,  P.  &  Caples,  M.),  the  covers  of  which  are  shown  in  Figure  5. 
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Figure  5.  Global  Horizon  Scan  Surveys 


IDENTIFICATION  AND  DOWN-SELECTION  OF  SYSTEMS 
JPM-TMT  RFI 

The  JPEO-CBD  JPM-TMT  (now  part  of  JPM-MCS)  office  issued  an  RFI,  "Development  of  a  Handheld 
Pathogen  Identification  and  Characterization  System,"  on  FedBizOpps.gov  that  sought  information 
regarding  existing  biological  agent  identifier  technologies.  The  solicitation  requested  information 
regarding  any  "lightweight,  easy  to  use,  handheld  bio-identification  system  to  provide  the  frontline 
warfighters  with  the  ability  to  identify  and  characterize  pathogens."  The  RFI  solicited  specific 
information  for  systems  that  perform: 

1.  Analysis  to  include  pathogen  characterization  and  identification,  and  the  potential  to 
identify  chemical  agents 

2.  Sample  preparation  to  be  used  prior  to  a  sample  analysis  on  a  handheld  device 

These  two  capabilities  would  ideally  function  as  an  integrated,  end-to-end  system  or  as  separate 
technologies  with  compatible  characteristics.  The  companies  were  asked  to  provide  details  on  their 
technologies,  in  the  form  of  these  inquiries: 

1.  Describe  the  technology  platform 

2.  What  is  the  multiplex  capability  of  the  technology? 

3.  What  assays  have  been  developed  for  the  platform  (e.g.,  infectious  disease  agents, 
biowarfare  agents)? 

4.  How  have  the  assays  been  tested  and  validated? 

5.  What  is  the  assay  run  time? 

6.  Does  the  device  have  the  capability  to  discriminate  between  biological  variants? 

7.  Does  the  device  identify  or  distinguish  threat  characteristics  (e.g.,  antibiotic  resistance, 
increased  virulence)? 

8.  What  is  the  sensitivity  and  specificity  of  the  assays? 

9.  How  much  sample  is  required  for  testing? 

10.  Is  sample  preparation  an  integrated  part  of  the  analysis?  What  are  the  requirements  for  the 
prepared  sample?  Is  the  raw  or  prepared  sample  preserved? 
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11.  What  samples  /  biologies  can  be  assayed  (e.g.,  spores/cysts,  Gram-positive  and  Gram¬ 
negative  bacteria,  viruses)? 

12.  What  types  of  matrices  can  be  analyzed  (e.g.,  blood,  sputum,  water,  surface  samples,  and 
aerosol  samples,  environmental)? 

13.  Is  the  technology  capable  of  identifying  chemical  warfare  agents,  toxic  industrial  chemicals, 
or  toxic  industrial  materials? 

14.  What  are  the  power  requirements  of  the  technology?  Is  it  a  mobile  device?  If  battery 
powered,  how  many  runs  per  single  battery  charge? 

15.  What  support  equipment  is  required?  Consumables?  Ancillary  sample  preparation 
equipment? 

16.  Under  what  environmental  conditions  has  the  technology  been  tested? 

17.  If  greater  than  3lbs,  how  much  development  time  would  be  needed  to  reduce  the  bio¬ 
identifier  to  less  than  three  lbs.  including  power  source? 

18.  How  is  the  analysis  data  presented  to  the  user?  Does  the  technology  have  data  transmission 
capability  (e.g.  internet,  satellite,  mobile  networks,  direct  interface  with  a  communications 
device)? 

19.  Does  the  technology  have  geolocation  capability? 

20.  What  is  the  current  unit  cost?  Sample  analysis  cost? 

21.  Does  the  device  require  calibration?  What  are  system  maintenance  requirements? 

Besides  the  companies  that  responded  to  the  RFI,  companies  with  potentially  applicable  technologies 
were  notified  of  this  opportunity  and  encouraged  to  submit.  Ultimately,  40  submissions  were  received, 
of  which  30  submissions  described  a  biological  agent  identifier  technology  developed  to  at  least  a 
"working  prototype"  level  of  maturity.  Eight  of  the  submissions  were  proposed  or  conceptual  designs 
and  two  were  exclusively  sample  preparation/collection  devices. 

Down  Selection  of  Candidate  Systems 

A  selection  panel  consisting  of  staff  from  ECBC  and  JPEO-CBD  JPM-TMT  reviewed  30  biological  agent 
identifier  system  submissions  to  the  RFI.  The  systems  represented  stages  of  technology  development 
from  advanced  concept  to  COTS  instruments.  The  reviewers  separated  candidate  systems  into  two 
categories  based  on  size  and  technology: 

•  Handheld  -  Antibody  and  Nucleic  Acid-based 

•  Mobile  Lab  -  Antibody  and  Nucleic  Acid-based 

The  RFI  responses  were  the  foundation  of  this  selection,  however,  other  relevant  sources  of  information 
and  systems  were  also  considered  (e.g.,  Emanuel,  P.  &  Caples,  M.  (2011)  The  Chemical  Biological 
Radiological  Technology  Survey).  Systems  were  selected  to  represent  a  diverse  collection  of 
identification  methodologies  with  a  focus  on  unique  technology  characteristics,  stage  of  development  of 
the  systems  or  the  time  needed  for  further  development,  assay  availability,  and  logistics.  After  further 
assessment  and  characterization,  12  candidate  systems  were  chosen  for  inclusion  in  the  formal  test  bed 
assessment.  Those  systems  and  their  associated  companies  are  shown  in  Tables  1  and  2. 
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Table  1.  Selected  Nucleic  Acid-Based  Biological  Detection  Identifiers 


B  I  O 


FIRE' 


Nested  PCR  with  melt  curve 
analysis;  sample  preparation 


PALLADIUM  SYSTEM 


PCR  and  hybridization  to 
complete  electrical  circuit 


LIAT™ 

ANALYZER 


Sample  preparation  and  real¬ 
time  PCR 


T-COR4™  REALTIME  PCR 


Real-time  PCR 
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Table  2.  Selected  Antibody-Based  Biological  Detection  Identifiers 


Fluorescently  labeled 
magnetic  microsphere 
capture  antibody 


CARTRIDGE  ^ 

READER  Pi  O0O 

Electrochemiluminescence 

Meso  Scale  Discovery 

(ECL) 

Antibody-based  wave  guide 
detection 


SPINDX™ 

V: 

r  1  Sandia 
Ifni  National 
*  J  Laboratories 

Antibody-based  capture 
beads 

Seattle  Sensor  Systems 

Surface  plasmon  resonance 
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Selection  of  System  Attributes  for  Evaluation 

The  selection  panel  held  several  discussions  to  select  relevant  information  and  performance  attributes 
to  be  captured  pertaining  to  each  biological  agent  identifier  system.  These  attributes  were: 

•  Single  Target  Identification 

•  Multiplex  Target  Identification 

•  Multiplex  Capability 

•  Assay  Flexibility 

•  Batch  Size 

•  RunTime 

•  Size 

•  Power 

•  Logistical  Support 

•  Costs 

•  Compatibility  and  Interchangeability 

•  Usability 

•  Maturity 

Attribute  information  was  collated  into  a  Biological  Identifier  Assessment  Table  (Appendix  B). 
Information  was  both  vendor-supplied  and  based  on  the  laboratory  assessment.  The  panel  scored 
system  attributes  based  on  objective  criteria,  where  possible.  Individual  attributes  for  each  system  were 
scored  on  a  scale  of  one  to  10  compared  to  a  theoretically  ideal  (i.e,  "perfect  ten")  technology  or 
product.  Each  attribute  category  also  received  a  weight  corresponding  to  its  importance  and  utility.  By 
multiplying  the  scores  and  weights,  the  various  biological  agent  identifiers  were  ranked  in  comparison  to 
ideal  instruments.  The  weighting  schemes  were  designed  to  be  scalable  so  individual  end-users  would 
be  able  to  adjust  weights  for  particular  attributes  based  on  the  end-user's  requirements,  allowing  them 
to  obtain  a  customized  ranking  of  technologies.  In  this  manner,  future  acquisitions  can  be  planned  by 
using  intelligent  decision-making  based  on  flexible  and  weighted  down-selection  criteria. 


LABORATORY  ASSESSMENT  OF  SYSTEMS 
Target  Panel 

A  goal  of  the  laboratory  assessment  was  to  directly  compare  the  systems'  ability  to  identify  common 
threats  and  their  versatility  to  identify  various  classes  of  biological  threat  agents.  The  specific  agents 
selected  to  represent  these  classes  include: 

•  Bacillus  anthracis  (Gram+  spore  forming  bacilli) 

•  Yersinia  pestis  (Gram-  rod-shaped  bacterium) 

•  Vaccinia  (VAC;  double-stranded  deoxyribonucleic  acid  (dsDNA)  Orthopox  virus,  Smallpox 
[Variola]  stimulant) 

•  Venezuelan  Equine  Encephalitis  (VEE;  +sense  single-stranded  RNA  virus,  an  Alphavirus) 

•  Clostridium  botulinum  Type  A  neurotoxin  (BoNT  A;  protein  toxin) 

Assay  Development 

Of  the  12  systems  that  were  down-selected  for  further  assessment,  eight  did  not  have  assays  available 
for  some  of  the  five  of  the  chosen  targets  in  the  Target  Panel.  Therefore,  ECBC  provided  support  for 
assay  development  to  the  manufacturers  of  the  following  devices: 
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1.  Epistem  Genedrive 

2.  INT  Palladium 

3.  IQuum  Liat 

4.  MesoScale  Diagnostics  (MSD)  Cartridge  Reader 

5.  Research  International  RAPTOR 

6.  Sandia  National  Laboratories  SpinDx 

7.  Seattle  Sensors  SPIRIT 

8.  Luminex  MAGPIX 

Where  necessary  to  develop  specific  assays,  antibodies  and  inactivated  agents 
were  provided  to  the  companies  by  the  Critical  Reagents  Program  (CRP),  a 
component  of  JPEO-CBD's  JPM-MCS.  Among  other  duties,  JPM-MCS  is 
responsible  for  research,  development  and  acquisition  of  U.S.  Food  and  Drug 
Administration  (FDA)-approved  medical  systems  for  diagnostic  capabilities 
against  biological  threat  agents. 

Nucleic  acid-based  biological  agent  identifiers  do  not  have  the  capability  to 
directly  identify  toxins  since  they  are  protein-based  agents.  However,  the  manufacturers  of  the 
Genedrive,  Liat,  and  Palladium  attempted  to  develop  immuno-PCR  assays.  While  none  of  these 
companies  successfully  developed  immuno-PCR  assays  for  their  systems,  Epistem,  with  a  commercial 
partner,  succeeded  in  developing  an  activity-based  BoNT  A  assay  for  use  on  the  Genedrive  platform.  This 
assay  utilizes  a  synthetic  fluorometric  substrate  to  directly  measure  the  presence  of  C.  botulinum  Type  A 
enzyme  proteolytic  activity. 

The  FilmArray,  RAZOR  EX,  and  T-COR  4  systems  were  not  able  to  directly  identify  toxins  and  did  not 
attempt  to  develop  assays  specific  for  BoNT  A.  However,  assays  were  available  for  the  instruments  to 
directly  detect  DNA  from  Clostridium  botulinum,  the  bacteria  that  produce  the  toxin.  Native  toxin 
preparations  may  contain  residual  DNA  from  the  source  organism;  therefore,  the  FilmArray,  RAZOR  EX 
and  T-COR  4  were  also  evaluated  for  the  capability  to  indirectly  detect  toxin  by  detecting  the  C. 
botulinum  bacterial  DNA. 

The  FilmArray,  RAZOR  EX,  and  T-COR  4  systems  had  assay  kits  that  were  well  established  and  did  not 
need  development.  The  FilmArray  had  assays  for  the  4  non-toxin  targets  while  the  RAZOR  EX  and  T-COR 
4  had  assays  for  Bacillus  anthracis  and  Yersinia  pestis,  but  not  VEE  or  VAC.  A  complete  list  of  assays 
available  for  each  instrument  is  shown  in  Table  4. 

Three  of  the  antibody-based  biological  agent  identifiers  can  detect  all  five  targets  in  multiplex  assays 
and,  therefore,  assay  development  was  not  necessary.  The  exceptions  were  the  RAPTOR,  which  did  not 
have  a  VEE  assay,  and  the  SPIRIT,  which  could  not  perform  multiplex  detection  using  the  current  analysis 
software  package.  Manufacturers  of  the  RAPTOR,  SPIRIT,  Cartridge  Reader,  and  SpinDx  were  provided 
antibodies  from  the  CRP  to  develop  assays.  The  NIDS  has  assays  for  all  five  targets,  but  they  are 
distributed  across  two  5-Plex  assay  cassettes.  Singleplex  and  multiplex  assays  for  the  Luminex  MAGPIX 
were  developed  by  ECBC  BioSensors  staff  using  CRP  antibodies.  The  MAGPIX  is  an  analyte 
capture/multiplex  bead  based  detection  system  that  has  the  capability  of  capturing  both  immunological 
and  molecular  events;  this  study  evaluated  only  the  immuno  capability. 
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Pre-Laboratory  Assessment  of  System  Specifications  and  Logistics 

The  systems  underwent  an  assessment  prior  to  the  laboratory  evaluation.  System  Evaluation 
Worksheets  (Appendix  C)  were  completed  for  each  system  and  contain  details  on: 

•  equipment  specifications 

•  shipment  of  equipment 

•  shipment  of  reagents 

•  set  up  of  equipment 

•  storage 

•  shelf  life 

Laboratory  Assessment  of  System  Performance 

Biological  Agents,  Nucleic  Acids,  and  Antibodies 

The  following  inactivated  (gamma  irradiated)  bacterial  and  viral  strains  were  purchased  by  ECBC  through 
the  CRP  and  used  in  the  laboratory  assessments: 

•  Bacillus  anthracis  Ames 

•  Yersinia  pestis  C092 

•  VAC  Elstree  (Lister) 

•  VEE  virus,  vaccine  strain  TC-83 

These  inactivated  bacteria  and  viruses  served  as  the  test  agents  for  the  particular  assays  of  interest  in 
antibody-based  bio-identifiers.  Nucleic  acids  were  derived  from  the  inactivated  bacteria  and  viruses 
either  through  on-board  or  external  sample  preparation.  This  study  used  gamma  irradiated  agents 
rather  than  live  cells  to  minimize  safety  risks  during  the  completion  of  mobile  laboratory  and  outdoor 
field-simulated  exercises.  Specifications  for  each  strain  can  be  found  in  Table  3. 


Table  3.  Specifications  of  Bacterial  and  Viral  Agents  Used  in  the  Laboratory  Assessments 


Agent 

Concentration  (Colony 
Forming  Units 
(CFU)/milliliter  (mL)  or 
Plaque  Forming  Units 
(PFU)/mL) 

Genomic 
Equivalen 
ts  (GE)1 

GE/CFU 

Ratio 

Estimated 

Genome  Size 
(Kb)2'3 

GE/nanogram 
(ng)  Nucleic 
Acid4 

B.  anthracis  Ames 

6.68x10s 

1.59xl09 

2.38 

5227 

1.77x10s 

Y.  pestis  C092 

3.01xl09 

4.17x10s 

1.39 

4830 

1.92X105 

VAC  Elstree  (Lister) 

1.31x10s 

1.89x10s 

1.44 

189 

4.90x10s 

VEE  virus,  TC-83 

l.OOxlO10 

lxlO10 

1.00 

11.4 

1.62x10s 

1  Data  from  CRP  Certificate  of  Analysis.  Genomic  equivalents  for  the  VEE  antigen  were  not  reported  on  the 


Certificate  of  Analysis.  For  the  purpose  of  this  study  a  ratio  of  1.00  was  used. 

2  Genomes  Online  (www.genomesonline.org)  Accession  Numbers  Gc00136  and  Gc00064  for  B.  anthracis  and 
Y.  pestis,  respectively 

3  GeneBank  Accession  Numbers  DQ121394.1  and  L01443.1  for  VAC  and  VEE,  respectively 

4  http://www.endmemo.com/bio/dnacopynum.php 

Antibodies  used  in  this  study  were  purchased  by  ECBC  through  the  CRP  and  used  in  the  laboratory 
assessment.  These  matched  pairs  have  previously  been  effectively  used  in  sandwich-type 
immunoassays.  The  Seattle  Sensors  SPIRIT  and  the  Research  International  RAPTOR  systems  only 
required  one  antibody  per  each  target.  The  antibodies  purchased  for  testing  were: 
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•  anti-8,  anthracis  monoclonal  antibody  (Cat  Num:  AB-BA-MAB4,  Lot  Num:  R0178) 

•  Goat  anti-8,  anthracis  antibody  (Cat  Num:  AB-G-BA,  Lot  Num:  PGGG016) 

•  anti-K  pestis  monoclonal  antibody  (Cat  Num:  AB-YERS-MAB1,  Lot  Num:  R0183) 

•  Rabbit  anti-K  pestis  antibody  (Cat  Num:  AB-R-YERS,  Lot  Num:  J040400-01) 

•  anti-VAC  monoclonal  antibody  (Cat  Num:  AB-VACC-MAB2,  Lot  Num:  J-191101-01) 

•  Rabbit  anti-VAC  antibody  (Cat  Num:  AB-R-VACC,  Lot  Num:  080205-01) 

•  anti-VEE  monoclonal  antibody  (Cat  Num:  AB-VEE-MAB2,  Lot  Num:  220711-01) 

•  anti-VEE  monoclonal  antibody  (Cat  Num:  AB-VEE-MAB3,  Lot  Num:  J-291002-01) 

•  anti-BoNT  A  monoclonal  antibody  (Cat  Num:  AB-BOT-A-MAB1,  Lot  Num:  030707-01) 

•  anti-BoNT  A  monoclonal  antibody  (Cat  Num:  AB-BOT-A-MAB2  Lot  Num:  260607-01) 

Toxin  Source 

BoNT  A  was  purchased  from  Metabiologics,  Inc.  (Madison,  Wl)  as  the  active  holotoxin  complex.  The 
concentration  of  the  toxin  was  lmg/mL  with  a  specific  toxicity  of  3.5xl07  (MLD50  /mg).  The  A260/278 
ratio  of  the  toxin  product  was  determined  by  the  producer  to  be  less  than  0.55,  indicative  of  a 
preparation  that  has  low  DNA  contamination. 

Sample  Preparation 

The  systems  processed  samples  from  one  of  these  sources: 

•  whole  inactivated  agent  (all  antibody-based  systems) 

•  inactivated  agent  processed  on-board  the  instrument  (FilmArray,  Liat)  prior  to  analysis,  or 
by  an  affiliated  sample  dilution  (RAZOR  EX) 

•  purified  DNA  via  Qiagen  DNeasy  (T-COR  4,  Genedrive) 

The  basic  test  sample  was  comprised  of  the  agent  in  pristine  buffer.  Several  systems  directly  analyzed 
this  type  of  test  sample  (FilmArray,  Liat,  NIDS,  Cartridge  Reader,  SPIRIT,  SpinDx,  MAGPIX,  and  Raptor). 
However,  RAZOR  EX  utilized  a  simple  sample  dilution  step  prior  to  loading  on  the  system,  while  the  T- 
COR  4  and  Genedrive  systems  required  sample  preparation  in  the  form  of  external  nucleic  acid 
purification. 
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Singleplex  Assays 

Each  of  the  biological  agent  identifiers  was  assessed  for  their  ability  to  identify  each  of  five  threat  agents 
prepared  in  separate  test  samples.  Some  manufacturers  developed  custom  assays  for  inclusion  in  this 
assessment.  However,  most  of  the  nucleic  acid  platforms  were  unable  to  directly  detect  BoNT  A  and 
other  systems  lacked  assays  for  one  or  more  targets  (Table  4). 


Table  4.  Singleplex  Assay  Study  Design  Used  in  the  Laboratory  Assessment 


ASSAY  AVAILABLE  FOR  TEST  AGENT 


SYSTEM 


B.  anthracis  Y.  pestis 


VAC 


VEE 


BoNT  A 
toxin 


Nucleic  acid- 

based 

FilmArray 

V 

V 

Genedrive 

S 

S 

S 

S 

Liat 

V 

- 

Palladium3 

- 

- 

- 

- 

- 

RAZOR  EX 

S 

- 

- 

y1 

T-COR  4 

S 

S 

- 

- 

Antibody- 

based 

Cartridge 

Reader 

S 

S 

S 

s 

MAGPIX 

S 

S 

S 

S 

NIDS  SAR  III 

S 

S 

S 

RAPTOR 

S 

S 

S 

- 

s 

SpinDx 

S 

S 

S 

s 

SPIRIT 

V 

S 

s 

1  The  system  has  an  assay  for  C.  botulinum,  the  bacterial  source  of  BoNT  A  toxin,  and  was  tested  for 


the  ability  to  detect  residual  host  DNA  in  the  BoNT  A  preparation. 

2  Epistem  developed  an  enzyme  activity-based  assay  for  BoNT  A. 

3  The  Palladium  device  did  not  arrive  in  time  to  be  included  in  this  evaluation. 


The  vendor-supplied  expected  limit  of  detection  (LOD)  was  the  basis  for  a  target  concentration.  The 
flowchart  in  Figure  6  demonstrates  how  vendor  LOD  claims  were  evaluated  during  the  laboratory 
assessment. 
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**lf  the  LOD  was  not  provided  by  the  vendor  testing  began  at  a  concentration  appropriate  for  the 
detection  method. 

Information  gathered  during  the  singleplex  and  multiplex  testing  in  ECBCs  stationary  laboratory  was 
recorded  in  a  System  Analysis  Report  (Appendix  C)  and  added  to  the  Biological  Identifier  Assessment 
Tables  (Appendix  B). 

Multiplex  Assays 

The  biological  agent  identifiers  were  assessed  for  their  ability  to  simultaneously  identify  up  to  five  threat 
agents  in  test  samples  containing  all  five  agents  (Table  5).  Of  the  12  instruments,  only  four  had  the 
capability  to  test  for  all  five  agents.  The  nucleic  acid-based  instruments  could  not  detect  BoNT  A  because 
it  was  a  protein,  while  some  instruments  did  not  have  assays  available  for  one  or  more  of  the  targets. 
The  empirically  determined  LOD  from  the  singleplex  assessment  was  the  basis  for  a  target 
concentration.  When  a  biological  agent  identifier  failed  to  detect  a  threat  agent  in  the  multiplex  format, 
additional  test  samples  were  prepared  at  concentrations  ten  times  above  this  target.  Positive  samples 
were  verified  by  triplicate  runs. 
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Table  5.  Multiplex  Assay  Study  Design  Used  in  the  Laboratory  Assessment 


SYSTEM  TEST  AGENTS 


Nucleic 

acid-based 

Film  Array 

B.  anthracis,  Y.  pestis ,  VAC,  VEE 

Genedrive 

B.  anthracis ,  Y.  pestis ;  VAC,  VEE 

Liat1 

B.  anthracis,  Y.  pestis ,  VAC,  VEE 

Palladium 

No  Assays 

RAZOR  EX 

B.  anthracis,  Y.  pestis 

T-COR  4 

N/A 

Antibody- 

based 

Cartridge  Reader 

B.  anthracis,  Y.  pestis ,  VAC,  VEE,  BoNT  A 

MAGPIX3 

B.  anthracis,  Y.  pestis,  VAC,  VEE,  BoNT  A 

NIDS  SAR  III1 

B.  anthracis,  Y.  pestis,  VAC,  VEE,  BoNT  A 

SpinDx 

B.  anthracis,  Y.  pesti,  VAC,  VEE,  BoNT  A 

SPIRIT2 

N/A 

RAPTOR 

B.  anthracis,  Y.  pestis,  VAC,  BoNT  A 

1  The  Liat  has  assays  for  four  targets  on  two  assays  while  the  NIDS  has  assays  for  all  five  of  the 
targets  on  two  5-plex  cartridges.  One  cartridge  contains  3  of  the  targets:  Y.  pestis,  VAC  and  VEE 
and  the  other  contains  2  of  the  targets:  B.  onthracis  and  BoNT  A  toxin. 


2  The  SPIRIT  is  functionally  capable  of  performing  multiplex  detection  of  up  to  three  targets  but 
the  current  analysis  software  can  only  determine  presence  of  target  in  1  of  the  3  available 
positions. 

3  The  MAGPIX  multiplex  detection  assay  did  not  undergo  optimization 
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FilmArray® 


by  BioFire™  Diagnostics 


System  Specifications 


Vendor:  BioFire  Diagnostics,  Inc. 

Website:  www.biofiredx.com 

System  Cost:  $49,500.00 
Assay  Cost:  $185 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  17  (BioThreat  Panel)  including  multiple 
targets  per  agent 

Assay  Shelf  Life:  4-6  months  at  room  temperature 

Sample  Size  Required:  250  pL 

Type  of  Detection:  Nucleic  acid  amplification  with  end  point 
melt-curve  analysis 

Time  to  Result:  65  minutes 

System  Weight:  20  lbs  (systems  requires  laptop  and  pouch 
preparation  station  not  included  in  weight) 

Operating  Range:  59-86  °  F  (15  -  30  °  C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab. 


System  Description 


The  BioFire  FilmArray  system  is  a  multiplex  PCR  system 
that  integrates  sample  preparation,  amplification,  detection 
and  analysis.  The  BioThreat  Panel  v2.4  pouch  stores  all 
the  necessary  reagents  for  sample  preparation,  RT- 
PCR,  PCR  and  detection  in  a  freeze-dried  format.  Once 
an  unprocessed  sample  is  injected  in  to  the  pouch  the 
FilmArray  will  extract  and  purify  nucleic  acids;  perform  an 
initial,  large  volume  multiplex  PCR;  and  complete  individual 
singleplex,  second-stage  PCR  reactions  to  identify  specific 
targets.  Finally  the  system  uses  meta  analysis  of  endpoint 
melt  curve  data  to  generate  a  result  for  each  agent  based 
on  the  results  of  one  or  more  targets,  each  performed 
in  triplicate.  Each  assay  contains  internal  standards  to 
automatically  control  for  each  step  of  the  process.  The 
design  of  this  system  requires  minimal  user  training  and 
very  little  hands  on  time. 


Test  Bed  Review 


The  FilmArrays’  ability  to  screen  17  different  pathogens 
(25  targets  total)  in  60  minutes  with  sensitivity  as  low 
as  5xl02  CFU/ml  makes  this  system  an  asset  in  any 
laboratory.  Although  the  FilmArray  is  not  “portable”,  the 
unique  BioThreat  pouch  had  the  ability,  during  our  study, 
to  simultaneously  detect  3  targets  of  Bacillus  anthracis , 

2  targets  of  Yersinia  pestis,  2  targets  of  VEE  virus  and 
2  targets  of  Orthopox  virus.  During  testing  the  targets 
were  run  as  singleplex  samples,  and  then  the  system  was 
challenged  using  multiple  targets  in  a  single  sample.  The 
FilmArray  was  able  to  detect  all  4  targets  with  no  loss  to 
sensitivity. 

During  the  mobile  laboratory,  scientists  were  impressed 
with  the  ease  of  use  that  the  pouch  and  its  associated 
stand  offered.  A  quick  10  minute  training  with  each 
end-user  allowed  them  to  run  the  system  independently 
with  one  scientist  commenting,  “I  wish  every  system  was 
this  easy  to  run”.  The  biggest  impact  the  system  had  on 
the  Mobile  Lab  scientists  was  its  multiplex  capability. 

“For  screening  an  unknown  sample,  this  system  would 
be  invaluable”,  one  commented.  With  time  not  being  an 
immediate  concern  for  this  group,  the  FilmArray  was  among 
their  top  picks  of  systems. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


JlNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

5.00xl02 

O 

5.00x10s 

Yersinia  pestis 

5-OOxlO1 

© 

5.00x10° 

Vaccinia 

l.OOxlO2 

O 

l.OOxlO3 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

O 

l.OOxlO6 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

5.00x10s 

© 

5.00x10s 

Yersinia  pestis 

5.00x10° 

O 

5-OOxlO1 

Vaccinia 

l.OOxlO3 

© 

l.OOxlO3 

Venezuelan  Equine 
Encephalitis 

l.OOxlO6 

© 

l.OOxlO6 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 

(0^2 - 3 - 4 - S-l 


MOBILE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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RAZOR® 


by  BioFire  Diagnostics,  Inc. 


System  Specifications 


Vendor:  BioFire  Diagnostics,  Inc. 

Website:  www.biofiredx.com 

System  Cost:  $38,500.00 
Assay  Cost:  $200.00 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  10 

Assay  Shelf  Life:  1  year  at  room  temperature 

Sample  Size  Required:  250  pL 

Type  of  Detection:  Nucleic  Acid 

Time  to  Result:  25  minutes 

System  Weight:  11  lbs 

Operating  Range:  32-104  °F(0-40  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field. 

°  d 


System  Description 


BioFire’s  RAZOR  EX  is  a  field  PCR  unit  that  uses  pouches 
pre-loaded  with  freeze-dried  PCR  reagents  for  the  detection 
and  identification  of  biological  pathogens  and  biothreat 
agents.  Each  kit  contains  all  of  the  items  necessary  for 
sampling,  sample  preparation  and  real-time  PCR.  Each 
kit  includes  items  needed  for  collecting  and  loading  the 
sample.  Once  samples  are  loaded  into  the  pouch  with 
cannula-tipped  syringes  they  are  dispensed  automatically 
in  to  the  wells,  requiring  no  precise  measuring.  BioFire 
has  assays  available  for  CDC  defined  Category  A  and 
B  Biothreat  pathogens.  One  pouch  will  test  for  3  to  10 
different  agents,  depending  on  configuration,  and  includes 
internal  controls  to  validate  the  integrity  of  the  test. 
BioFire’s  pouches  and  its  associated  kit  components  are 
manufactured  under  a  cGMP  quality  system.  This  testing 
used  The  TEN™  10  Target  Screen  Kit  to  detect  2  out  of 
4  desired  targets,  Bacillus  anthracis  and  Yersinia  pestis, 
and  verify  vendor  claims  of  limits  of  detection  (LOD). 
Additionally,  the  Botulinum  toxin  preparation  was  tested  for 
residual  C.  botulinum  DNA. 


Test  Bed  Review 


The  RAZOR  was  one  of  the  few  systems  in  this  test  that  had 
been  designed  as  a  field-ready  system.  Because  of  this,  the 
expectation  that  it  would  outperform  the  others  was  there, 
however  the  ease  of  use  did  not  meet  up  with  the  end 
users  requirements. 

In  the  laboratory  the  system  performed  comparable  to  the 
other  PCR  systems,  with  LOD’s  down  to  102,  for  its  Yersinia 
pestis  assay.  Unfortunately,  BioFire’s  RAZOR  is  considered 
a  “closed”  system,  the  company  was  not  willing  to  develop 
new  assays  for  this  study  therefore  only  2  ( Yersinia  pestis 
and  Bacillus  anthracis)  of  the  desired  5  targets  were 
tested.  This  closed  system  is  an  unfavorable  characteristic 
for  military  applications  that  are  challenged  with  new 
threats  and  require  new  assays  at  any  given  time. 

During  the  field  testing,  soldiers  had  difficulty  manipulating 
the  pouch  and  found  the  barcode  scanner  difficult  to 
use.  One  commented,  “This  system  is  tedious”.  Another 
concern  for  the  soldiers  in  the  field  was  the  difficulty  they 
had  reading  the  screen  in  the  daylight.  One  soldier  noted 
“the  brightness  option  does  not  help  in  the  bright  sunlight”. 
Overall  the  field  ready  system  isn’t  as  ready  as  it  appears. 

In  full  MOPP  gear  the  consumables  gave  the  greatest 
challenge  even  for  the  simplest  of  tasks,  such  as  opening 
a  box. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


JlNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

l.OOxlO3 

1.30xl04 

Yersinia  pestis 

l.OOxlO2 

O 

1.30xl03 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

1.30xl04 

© 

1.30xl04 

Yersinia  pestis 

1.30xl03 

© 

1.30xl03 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


Q  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 

vi  i  T  3 

MOBILE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Genedrive™ 


by  Epistem,  Inc. 


System  Specifications 


Vendor:  Epistem,  Inc. 

Website:  www.epistem.co.uk 

System  Cost:  $4,000 

Assay  Cost:  Price  Request  (Price  estimated  to  be  $85/assay, 
subject  to  quantity) 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  4  per  Assay 

Assay  Shelf  Life:  Unknown 

Sample  Size  Required:  20  pL 

Type  of  Detection:  Nucleic  Acid 

Time  to  Result:  60  minutes 

System  Weight:  1.2  lbs 

Operating  Range:  32-131  °F  (0-55  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  is  still  in  prototype  stage.  Some  components 
need  refinement. 


System  Description 


The  Genedrive  is  a  fully  integrated,  endpoint  PCR-based 
platform.  Genedrive’s  proprietary  “hybrid  thermal  engine” 
allows  faster  cooling  rates  and  shorter  annealing  times 
relative  to  conventional  PCR  devices.  The  system  is  capable 
of  performing  ultra  fast  PCR  cycling  of  30  cycles  in  as  little 
as  17  minutes  and  is  controlled  by  a  single  button.  The 
Genedrive  was  designed  to  be  a  highly  cost-effective  way 
of  moving  molecular  diagnostics  from  the  laboratory  to  the 
point  of  need  across  several  markets  including  government. 
Epistem  currently  has  a  Mycobacterium  tuberculosis 
IVD  assay  for  the  Genedrive  that  has  received  European 
approval.  For  this  testing  Epistem  developed  multiplex 
assays  to  detect  Bacillus  anthracis,  Yersinia  pestis, 

Vaccinia  and  Venezuelan  Equine  Encephalitis  and 
a  singleplex  assay  to  detect  active  Botulinum  neurotoxin 
type  A. 


Test  Bed  Review 


On  paper,  the  Genedrive’s  low  cost,  small  footprint  and 
short  run  time  were  appealing;  unfortunately,  the  assays 
did  not  meet  expectations  and  required  modified  testing 
procedures  to  achieve  successful  data.  After  several 
iterations,  including  eliminating  the  sample  preparation 
paper  and  making  internal  hardware  structure  changes,  the 
system  was  able  to  detect  approximately  1.66xl05 

CFU/mL  in  the  Yersinia  pestis  assay.  However,  Bacillus 
anthracis  and  Vaccinia  detection  limits  were  100  fold 
higher  and  the  device  couldn’t  run  the  VEE  assay  because 
of  firmware  shortcomings.  The  60  minute  run  time  was 
unexpected,  since  the  system  was  projected  to  perform 
30  cycles  in  17  minutes.  Genedrive  also  produced 
false  positives  and  inconsistent  results.  Our  testing 
demonstrated  that  unrefined  sample  preparation  and  assay 
reagents  also  contributed  to  poor  performance. 

Field-users  were  impressed  with  the  size  of  the  system, 
but  little  else.  Surprisingly,  the  sample  preparation  created 
the  hardest  task  of  manipulating  several  tubes.  One  user 
commented,  “Anything  that  requires  several  steps  would 
require  two  operators  for  set  up  and  therefore  would  not 
be  used  in  the  field”.  One  soldier  noted  that  the  humid 
weather  prevented  the  sample  preparation  cards  from 
completely  drying.  Another  user  commented  “The  device 
needs  some  method  of  data  accessibility  and  storage  such 
as  a  thumb  drive  for  chain  of  evidence.” 

Genedrive  did  not  perform  well  for  bacterial  and  virus 
detection  but  was  the  only  PCR-based  system  that 
could  detect  the  presence  of  toxin.  The  system  design  is 
promising  and  grabs  one’s  attention,  but  assay  design  and 
sample  preparation  require  additional  development,  putting 
this  system  near  the  bottom  for  performance  in  our  testing. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  GE,  Virus  =  GE,  toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

10 

O 

69,000 

Yersinia  pestis 

10 

O 

200 

Vaccinia 

10 

O 

34,000 

Venezuelan  Equine 
Encephalitis 

10 

O 

Not  Tested 

Clostridium 

Botulinum  Toxin 

No  Claim 

O 

10,000 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

69,000 

O 

N/A 

Yersinia  pestis 

200 

O 

N/A 

Vaccinia 

34,000 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

10,000 

O 

N/A 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Liat™ 


by  IQuum 


System  Description 


The  Liat  is  an  automated  sample-to-result  detection 
analyzer.  IQuum’s  lab-in-a-tube  (Liat)  was  designed  to 
enable  non-specialized  personnel  to  perform  “moderate 
complexity”  tests  in  hospital  labs  or  other  near-patient 
setting.  The  Liat  has  automated  sample  processing  in  a 
flexible  tube  containing  pre-packaged  reagents.  Peristaltic 
manipulations  by  actuators  in  the  analyzer  move  the 
sample  through  each  stage  of  sample  processing  ending 
with  a  PCR  or  RT-PCR  amplification  the  sample  for 
target  identification.  Currently  IQuum  has  a  FDA  510(K) 
cleared  Liat  Influenza  A/B  assay  on  the  market  which  is 
intended  for  use  in  laboratories  certified  under  the  Clinical 
Laboratory  Improvement  Amendments  (CLIA).  For  this 
testing  IQuum  developed  two  duplex  assays,  one  that 
detects  Bacillus  anthracis  and  Yersinia  pestis  and  another 
to  detect  Vaccinia  and  Venezuelan  Equine  Encephalitis. 


System  Specifications 


Vendor:  IQuum 
Website:  www.iquum.com 
System  Cost:  $25,000.00 

Assay  Cost:  Price  on  Request  (Estimated  to  be  $60  per  test) 

Assay  Storage  Requirements:  Refrigeration 
Agents  Tested  per  Assay:  2 
Assay  Shelf  Life:  1  year  at  4  °C 
Sample  Size  Required:  200  pL 

Type  of  Detection:  Real-time  nucleic  acid  amplification  and 
fluorescence  detection 

Time  to  Result:  30  minutes 

System  Weight:  8.3  lbs 

Operating  Range:  40  - 122  °F  (4  -  50  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field. 

U  ?  .a.  a  ^ 


Test  Bed  Review 


A  combination  of  small  footprint,  automated  sample  prep, 
20  minute  run  and  sensitivity  down  to  the  single  digits 
(CFU/mL),  the  Liat  analyzer  was  a  top  performer  both  in 
the  lab  and  in  the  field.  Achieving  detection  levels  down  to 
6.5  CFU/mL  in  their  Yersinia  pestis  duplex  assay,  the  Liat’s 
sensitivity  was  superior  to  most  other  PCR  platforms  in  the 
test  bed.  In  addition  to  performing  well  in  the  presence 
of  one  agent,  IQuum  also  scored  well  in  the  multiplex 
category,  showing  little  to  no  loss  of  signal  when  samples 
were  combined  in  a  duplex  format.  During  testing,  the 
Liat  did  present  with  several  error  messages  that  required 
IQuum’s  intervention.  However,  these  errors  were  a  result 
of  system  checks  put  in  place  as  a  requirement  for  a  FDA 
approved  system. 

In  the  field,  the  end-users  were  impressed  with  the  ease 
of  set-up  and  minimum  amount  of  training  required.  One 
operator  stated  that  “This  device  is  the  easiest  piece 
of  equipment  I’ve  ever  used.”  Admittedly  the  current 
configuration  of  this  system  was  not  intended  for  outdoor 
use  and  end-users  would  like  to  see  some  modifications  to 
its  current  design.  The  size  of  the  buttons  and  the  use  of 
the  stylus  were  top  on  their  list.  One  user  commented  “The 
Login/Pin  requirement  is  overkill  for  military  applications 
and  the  stylus  would  get  lost”. 

As  a  result  of  testing  in  an  analytical  and  mobile  laboratory 
and  the  field,  the  Liat  showed  a  great  deal  of  potential. 

We  believe  with  little  investment  in  its  design  (i.e., 
ruggedization  of  the  exterior,  integrated  battery)  this  system 
would  be  a  good  fit  in  most,  if  not  all,  military  testing 
applications. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


JlNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

2.00X101 

Q 

l.OOxlO3 

Yersinia  pestis 

5.00x10° 

© 

6.5x10° 

Vaccinia 

2.00xl02 

© 

2.50xl03 

Venezuelan  Equine 
Encephalitis 

4.00xl04 

© 

2.10xl03 

Clostridium 

Botulinum  Toxin 

N/A 

o 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO3 

© 

l.OOxlO3 

Yersinia  pestis 

5.00x10° 

© 

l.OOxlO1 

Vaccinia 

2.50xl03 

© 

2.50xl04 

Venezuelan  Equine 
Encephalitis 

2.10xl03 

© 

2.10xl04 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


<u 

cn 

c 

uo 
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T-COR4™ 


by  Tetracore,  Inc. 


System  Specifications 


Vendor:  Tetracore,  Inc. 

Website:  www.tetracore.com 

System  Cost:  $38,500.00 
Assay  Cost:  $16,000.00 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  One.  Multiple  formats  available 

Assay  Shelf  Life:  1  year  at  room  temperature 

Sample  Size  Required:  3  pL 

Type  of  Detection:  Nucleic  Acid 

Time  to  Result:  45  minutes 

System  Weight:  6.2  lbs  (w/o  required  centrifuge) 

Operating  Range:  39-113  °F  (4-50  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field. 


System  Description 


The  T-COR  4  is  a  portable  Real-Time  PCR  thermocycler 
with  four  independent  sample  wells  capable  of  2  color 
detection.  The  T-COR  4  is  a  field  deployable  battery 
powered  system,  but  lacks  sample  preparation  capabilities. 
Tetracore’s  real-time  PCR  reagents  are  stored  at  room 
temperature  with  20+  assays  currently  available.  The 
system  in  encased  in  a  heavy  duty  protective  rubber  sleeve 
with  an  internal  8  hour  battery  for  the  thermal  cycler.  For 
this  testing,  Tetracore’s  Bacillus  anthracis  pXOl  assay 
and  Yersinia  pestis  assay  were  evaluated.  Each  assay 
contains  reagents  for  specific  target  detection  using  the 
FAM  fluorophore  and  an  internal  control  detected  with  CY5 
fluorophore. 


Test  Bed  Review 


The  T-COR  4  is  small,  lightweight  and  battery  operated, 
making  it  appealing  for  the  field.  The  main  shortcomings 
were  the  number  of  targets  (1)  per  test  and  lack  of 
integrated  sample  preparation.  With  purified  samples,  the 
system  scores  high  in  the  laboratory,  sensitivities  in  the 
femtogram  range.  The  20  minute  runtime  and  real  time 
viewing  are  appealing,  unfortunately  without  on  board 
sample  preparation,  the  scores  are  lower  than  the  other 
systems.  Additionally,  Tetracore  was  only  able  to  provide 
2  of  the  5  requested  targets,  with  no  ability  to  design 
new  assays  for  this  testing.  The  test  results  could  not  be 
saved  on  the  device,  as  configured.  Tetracore  can  provide 
software  to  operate  the  instrument,  and  save  and  analyze 
data  using  an  external  computer. 

In  the  field,  the  end-users  liked  the  ease-of-use  of 
the  system  and  had  no  trouble  running  this  system 
independently.  Although  one  soldier  commented,  “It 
doesn’t  really  add  anything  additional”,  another  added, 

“It’s  quick  to  start  up  and  easy  to  use”.  End  users  liked 
that  the  results  were  easy  to  view  and  interpret.  Consistent 
results  were  an  additional  plus  of  this  system.  Training  time 
was  minimal  and  the  consumables  were  easy  to  handle  in 
MOPP  gear.  The  small  centrifuge  for  the  PCR  tubes  added 
to  the  footprint  for  field  operators. 

As  a  result  of  testing  in  an  analytical  and  mobile  laboratory 
and  the  field  the  T-COR  4  was  a  consistent  and  easy  to 
use  system.  The  lack  of  onboard  sample  preparation  is  a 
negative,  but  the  system  is  a  reliable  workhorse. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  fg/uL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

No  Claim 

O 

3.2X101 

Yersinia  pestis 

No  Claim 

O 

1.3X101 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

3.2x10 1 

O 

N/A 

Yersinia  pestis 

1.3X101 

O 

N/A 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 


Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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NIDS® 


by  ANP  Technologies 


System  Specifications 


Vendor:  ANP  Technologies 
Website:  www.anptinc.com 

System  Cost:  $6,500.00 
Assay  Cost:  $45.00 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  5  per  Assay,  Multiple  formats  available 

Assay  Shelf  Life:  2  years  from  receipt  at  room  temperature 

Sample  Size  Required:  100-200  pL 

Type  of  Detection:  Antibody 

Time  to  Result:  15  minutes 

System  Weight:  1.6  lbs 

Operating  Range:  40-122  °F  (4-50  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  NIDS  assay  used  with  Stand  Alone  Reader  had 
successful  operation  in  Mobile  lab  and  field. 


System  Description 


ANP  has  developed  the  Nano  Intelligent  Detection  System, 
or  NIDS.  The  NIDS  is  a  multiplexed  Handheld  Assay  (HHA) 
together  with  a  palm-sized,  portable,  ruggedized  optical 
Stand  Alone  Reader  (SAR  III).  The  NIDS  technology  uses  an 
antibody  nanomanipulation  technique  that  orients  each 
antibody  so  that  there  are  optimal  biosensing  regions 
available  for  antigen  binding  and  sandwich  formation. 
Because  of  this  nanomanipulation,  the  NIDS  is  the  first 
HHA  that  claims  to  have  no  “Hook  Effect”.  The  NIDS  was 
designed  to  take  all  of  the  guess  work  out  of  interpreting 
HHA  results  by  different  responders  in  field  conditions  with 
poor  lighting  and  limited  visibility  while  wearing  personal 
protective  equipment. 


Test  Bed  Review 


The  NIDS  is  small,  rapid  and  has  the  ability  to  detect 
bacterial,  viral,  and  protein  toxins.  However,  it  still  delivers 
below  average  sensitivity  and  high  false  positive  rates  that 
we’ve  come  to  expect  with  a  typical  HHA.  Detection  levels 
ranging  from  lxlO6  to  lxlO8  CFU/mL  for  a  singleplex  assay 
and  lxlO6  to  lxlO9  CFU/mL  in  the  multiplex,  gave  the  NIDS 
an  overall  low  score  in  the  Target  Identification  Category  of 
the  Criteria  Table. 

Where  the  NIDS  did  fair  well  is  ease  of  use  and  small  size. 
Little  to  no  sample  preparation  time  is  required  for  this 
system,  maintenance  time  is  low  and  the  overall  sample  to 
result  is  15  minutes.  These  attributes  yielded  scores  of 
70-100%  in  4  of  the  5  categories. 

In  the  field,  the  end-users  were  impressed  with  the  ease 
of  set-up  and  minimum  amount  of  training  required. 
However  one  operator,  familiar  with  typical  HHAs  remarked 
“The  reader  makes  it  easier  than  current  HHAs,  but  it’s 
still  an  HHA.”  In  addition,  one  user  experienced  false 
positives  while  running  the  device,  which  was  quickly  fixed 
after  cleaning  the  lens.  However  this  result  did  lead  to 
concerns  of  contamination.  On  a  positive  note,  one  soldier 
commented,  “The  device  is  small  enough  to  fit  into  the 
pouch  of  my  body  armor”. 

As  a  result  of  testing  in  an  analytical  and  mobile 
laboratory  and  in  the  ECBC  Skippers  Point  Site  the  NIDS 
overall  performance  was  mediocre  at  best.  As  one  of 
the  only  currently  fielded  systems  in  our  test  bed,  better 
results  were  expected  from  this  system  and  users  were 
disappointed  by  the  recurring  false  positives.  Clearly  there 
is  a  need  for  a  more  sensitive  and  accurate  identification  of 
biothreats 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


|plNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

l.OOxlO6 

O 

l.OOxlO7 

Yersinia  pestis 

2.50x10s 

O 

2.50xl06 

Vaccinia 

l.OOxlO6 

O 

>1.00xl08 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

O 

l.OOxlO9 

Clostridium 

Botulinum  Toxin 

5-OOxlO1 

© 

5-OOxlO1 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO7 

© 

l.OOxlO7 

Yersinia  pestis 

l.OOxlO6 

© 

l.OOxlO6 

Vaccinia 

>1.00xl08 

© 

l.OOxlO8 

Venezuelan  Equine 
Encephalitis 

l.OOxlO9 

© 

l.OOxlO9 

Clostridium 

Botulinum  Toxin 

5-OOxlO1 

O 

5.00xl02 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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MagPix™ 


by  Luminex,  Corp 


System  Specifications 


Vendor:  Luminex,  Corp. 

Website:  www.luminexcorp.com 

System  Cost:  $24,000.00 

Assay  Cost:  Price  on  Request 

Assay  Storage  Requirements:  Refrigeration 

Agents  Tested  per  Assay:  Up  to  50 

Assay  Shelf  Life:  User  determined 

Sample  Size  Required:  20-200  pL 

Type  of  Detection:  Antibody 

Time  to  Result:  90-105  minutes  for  No  Wash  Assay 

System  Weight:  38.5  lbs 

Operating  Range:  50-104  °F  (10-40  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S. 

DoD’s  ASD  (R&E).  Assay  integration  requires  development  and 
optimization. 

*  ^-7  g  aj 


System  Description 


The  Luminex  MAGPIX  utilizes  labeled  magnetic  beads  in 
a  fluidics  system  combined  with  optical  detection  and 
computerized  analysis  to  perform  plate-based  multiplex 
immunoassays.  Preparation  of  the  sample(s)  is  done  in 
a  96-well  plate  on  the  benchtop  or  in  a  biological  safety 
cabinet  and  consists  of  multiple  incubations  with  antibody- 
and  fluorescently-labeled  magnetic  beads.  The  operator 
sets  assay  parameters  in  xPONENT  software  on  a  computer 
that  controls  the  MAGPIX  and  loads  the  assay  plate  to  the 
MAGPIX,  which  runs  the  analysis  of  the  prepared  samples. 
The  MAGPIX  can  run  highly  multiplexed  assays  to  measure 
up  to  50  different  analytes  simultaneously  in  a  sample. 

The  MAGPIX  also  has  capability  to  perform  nucleic  acid 
detection  utilizing  hybridizations  to  nucleic  acid-labeled 
magnetic  beads  and  has  developed  a  prototype  sample 
preparation  cartridge  that  would  automate  sample 
preparation.  For  this  testing,  all  assays  were  developed  by 
the  laboratory  scientists  at  ECBC. 


Test  Bed  Review 


MAGPIX  had  one  of  the  largest  footprints  of  the  biological 
identifiers,  including  computer,  magnetic  bead  separator 
and  a  plate  shaker.  Despite  the  utilization  of  a  faster 
“no-wash”  ELISA-type  assay  format,  the  assays  still  took 
approximately  2  hours.  The  software  required  training 
and  practice  to  become  comfortable  at  running  assays 
and  analyzing  the  data.  MAGPIX  assays  utilized  the  same 
antibodies  as  other  systems  in  this  assessment,  yet  were 
more  sensitive  than  most  other  devices.  Limits  of  detection 
were  approximately  105  CFU/  mL  sample  for  bacteria 
assays,  107  PFU/mL  sample  for  virus  assays  and  1  ug/mL 
for  toxin  assays. 

Because  of  its  size,  the  MAGPIX  was  assessed  in  the 
Mobile  Laboratory,  but  not  the  field  setting.  Mobile  lab 
end-users  gave  mostly  split  opinions  (among  FAIR,  GOOD 
and  EXCELLENT  scores)  on  the  MAGPIX.  The  exception  was 
for  “Ease  of  Use”  which  rated  FAIR,  since  this  instrument 
requires  some  training  and  more  sample  preparation 
than  most  other  biological  agent  identifiers.  One  operator 
with  flow  cytometry  experience  was  comfortable  with  the 
MAGPIX  and  affirmed,  “The  MAGPIX  is  similar  to  other 
Luminex  devices.”  The  perceived  safety  was  ranked  either 
FAIR  or  GOOD  due  to  operators  concerns  over  handling 
the  96-well  plates  with  possible  pathogens.  Users  felt  they 
could  minimize  the  safety  concerns  by  extensive  use  of 
bleach  and  the  biological  safety  cabinet  within  the  Mobile 
laboratory.  The  operators  suggested,  “The  MAGPIX  should 
be  used  as  a  confirmatory  technology,  after  an  initial 
positive  PCR  assay.” 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


|plNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

N/A 

O 

1.00x10s 

Yersinia  pestis 

N/A 

O 

1.00x10s 

Vaccinia 

N/A 

O 

l.OOxlO7 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

1.00x10s 

Clostridium 

Botulinum  Toxin 

N/A 

O 

l.OOxlO3 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

1.00x10s 

O 

l.OOxlO7 

Yersinia  pestis 

l.OOxlO5 

O 

l.OOxlO6 

Vaccinia 

l.OOxlO7 

O 

>1.00xl08 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

© 

l.OOxlO8 

Clostridium 

Botulinum  Toxin 

l.OOxlO3 

© 

l.OOxlO3 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Cartridge  Reader 


by  Meso  Scale  Discovery  (MSD) 


System  Specifications 


Vendor:  Meso  Scale  Discovery  (MSD) 
Website:  www.meso-scale.com 
System  Cost:  Price  on  Request 
Assay  Cost:  Price  on  Request 

Assay  Storage  Requirements:  Refrigeration 

Agents  Tested  per  Assay:  5 

Assay  Shelf  Life:  1  year  at  4  °C 

Sample  Size  Required:  165  |jL 

Type  of  Detection:  Antibody 

Time  to  Result:  30  minutes 

System  Weight:  13  lbs 

Operating  Range:  59-86  °F  (15-30  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field.  Prototype/Beta  test  unit. 

7  g  -a- 


System  Description 


The  Cartridge  Reader  is  one  of  the  few  non- 
COTS  systems  included  in  this  testing.  MesoScale 
Diagnostics’  (MSD)  Cartridge  Reader  uses  Multi-Array 
electrochemiluminescence  (ECL)  technology  to  provide 
highly-sensitive  multiplexed  detection  in  a  small  volume  of 
liquid  sample.  The  instrument  carries  out  measurements 
using  single-use  injection-molded  fluidic  cartridges  that  can 
conduct  multiplexed  measurements  of  up  to  12  targets 
in  a  sample.  Each  cartridge  has  integrated  positive  and 
negative  controls.  Integrated  fluidics  on-board  the  cartridge 
allow  for  fully  automated  sample  processing  and  analysis. 


Test  Bed  Review 


As  a  top  performer  and  end  user  favorite,  the  Cartridge 
Reader  was  easy  to  use,  featured  multiplex  capability  and 
consistently  performed  throughout  the  entire  testing. 

Achieving  detection  levels  down  to  lxlO5  cfu/ml  in  both  its 
Bacillus  anthracis  and  Yersinia  pestis  singleplex  assays, 
the  Cartridge  Readers’  sensitivity  was  comparable  to  other 
fully  developed  antibody-based  system.  In  addition  to 
performing  well  in  their  singleplex  assay  design,  MSD  also 
scored  well  in  our  “Multiplex”  category.  This  technology  was 
able  to  detect  all  five  targets  with  no  loss  of  signal  when 
combined  in  a  duplex  format.  During  the  initial  laboratory 
testing,  the  user  interface  displayed  raw  data  and  required 
exporting  the  data  to  an  EXCEL  macro  file  to  calculate  the 
final  results. 

Prior  to  field  testing,  MSD  was  contacted  and  asked  to 
modify  the  user  interface  so  that  results  were  easily 
interpretable  by  the  end  user.  MSD  was  able  to  perform  a 
firmware  upgrade  and  testing  within  one  week.  In  the  field, 
the  end-users  were  impressed  with  the  ease  of  set-up  and 
minimum  amount  of  training  required.  Soldiers  referred  to 
the  system  as  “Superior”  and  a  “Favorite”  of  the  antibody 
based  systems.  One  end-user  commented  that  the  system 
seemed  “almost  too  easy”.  Where  this  system  lacks  the 
sensitivity  of  a  PCR-based  technology  it  excels  in  its  ability 
to  analyze  up  to  12  different  agents  in  one  sample. 

Admittedly  the  current  configuration  of  this  system  was 
not  intended  for  outdoor  use  because  it  offers  no  battery 
option  and  the  system  has  not  been  ruggedized.  However 
these  characteristics  were  minor  concerns  for  the  end- 
users  who  viewed  this  system  as  one  of  their  top  picks. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


|plNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

l.OOxlO5 

© 

1.00xl05 

Yersinia  pestis 

l.OOxlO7 

© 

1.00x10s 

Vaccinia 

1.00x10s 

© 

l.OOxlO7 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

© 

1.00x10s 

Clostridium 

Botulinum  Toxin 

7.00xl0'2 

O 

l.OOxlO2 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO5 

© 

l.OOxlO5 

Yersinia  pestis 

l.OOxlO5 

© 

l.OOxlO5 

Vaccinia 

l.OOxlO7 

© 

>1.00xl08 

Venezuelan  Equine 
Encephalitis 

l.OOxlO8 

© 

1.00x10s 

Clostridium 

Botulinum  Toxin 

l.OOxlO2 

© 

l.OOxlO3 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 


MAN  PORTABLE 


MOBILE 

c 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 

100 


cn 

Ln 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  o 


VERY  LIMITED/ 
DIFFICULT 
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RAPTOR™  Plus 


by  Research  International,  Inc. 


System  Specifications 


Vendor:  Research  International,  Inc. 
Website:  www.researchintl.com 

System  Cost:  $49,500.00 
Assay  Cost:  $150 

Assay  Storage  Requirements:  Refrigeration 
Agents  Tested  per  Assay:  4 
Assay  Shelf  Life:  1  year  at  4  °C 
Sample  Size  Required:  1-2  ml_ 

Type  of  Detection:  Antibody 
Time  to  Result:  28  minutes 
System  Weight:  14.6  lbs  (with  battery) 

Operating  Range:  34  -95  °  F  (1  -  35  °  C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  completed  but  had  recurring  technical  issues. 

Cl  ,?  „  fl  -^^-7  g  q  V 


System  Description 


The  RAPTOR  by  Research  International  is  a  field-ready 
ruggedized  fluorometric  assay  system  about  the  size  and 
weight  of  a  car  battery  that  can  be  used  to  detect  biological 
agents,  chemical  contaminants  or  explosives.  For  biological 
agent  identification,  the  RAPTOR  uses  a  four  channel  wave¬ 
guide  system  with  specific  capture  antibodies  bound  to  an 
immunoassay  “coupon”  for  detection  and  identification  of 
potential  threat  agents.  The  operator  must  prepare  and 
emplace  the  detector  antibody  tubes,  a  pouch  with  running 
buffer  and  a  waste  pouch  prior  to  operation.  The  RAPTOR 
has  two  internal  peristaltic  pumps  that  control  the  fluids’ 
movements  and  the  assay  progression.  The  total  time  for 
set-up,  system  test  and  establishment  of  a  pre-operational 
baseline  reading  is  approximately  30  minutes,  while  the 
sample  run  time  is  14  minutes.  The  instrument  is  not 
quantitative;  however,  detection  signals  are  displayed  as 
negative ,  suspect ,  positive,  and  highly  positive  results, 
providing  some  indication  of  relative  quantity  of  a  particular 
target. 


Test  Bed  Review 


The  laboratory  assessment  for  the  RAPTOR  utilized  its 
Bioassay  Coupon  kit,  which  included  assays  for  Bacillus 
anthracis,  Yersinia  pestis,  Vaccinia  virus  and  BoNT  A. 

The  technical  staff  at  Research  International  was  not 
able  to  develop  a  working  assay  for  Venezuelan  Equine 
Encephalitis  virus.  The  RAPTOR  did  not  fare  well  in  the 
laboratory  assessment  as  positive  results  were  not 
obtained  at  all  for  Bacillus  anthracis  or  Vaccinia  virus  at 
up  to  lOOx  the  claimed  LOD,  while  Yersinia  pestis  was 
only  positive  at  5  x  107  CFU/mL,  or  lOx  the  claimed  LOD. 
The  RAPTOR  required  some  practice  to  become  adept  at 
making  the  proper  fluidics  connections  prior  to  operation. 
The  tubing  was  prone  to  kink  and  color  coded  connectors 
were  mismatched.  Also,  an  interior  module  that  holds  the 
detector  antibody  tubes  had  to  be  frozen  prior  to  operation, 
possibly  causing  inconsistent  assay  conditions  as  the 
instrument  warmed  throughout  the  day. 

The  RAPTOR  was  assessed  in  the  Mobile  Laboratory  and  by 
both  field  operator  groups.  However,  5  of  the  8  operators 
were  unable  to  complete  sample  analysis  because  the 
RAPTOR  failed  the  fluidics  pre-operational  testing  and 
would  not  operate  correctly,  or  failed  during  the  sample 
analysis.  Consequently,  the  operators’  rating  of  “Ease  of 
Use”  varied.  The  manipulation  of  tubing  and  pouches  prior 
to  sample  analysis  was  difficult  for  operators  in  multiple 
glove  layers.  One  operator  reflected,  “You  shouldn’t  have 
to  be  a  mechanic  to  set-up  the  device.”  The  RAPTOR 
received  consistently  positive  ratings  for  data  viewing  and 
interpretation  because  the  surface  display  is  simple  to  read 
and  understand. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

5.ooxio4- 

s.ooxio5 

O 

>1.00xl07 

Yersinia  pestis 

No  Claim 

O 

5.00xl07 

Vaccinia 

l.OOxlO5 

O 

>1.00xl07 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

1-10 

O 

l.OOxlO4 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

>1.00xl07 

O 

>5.00xl06 

Yersinia  pestis 

5.00xl07 

© 

5.00xl07 

Vaccinia 

>1.00xl07 

O 

>1.00xl07 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


100 


U1 

Ln 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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SpinDx™ 


by  Sandia  National  Laboratory 


System  Specifications 


Vendor:  Sandia  National  Laboratory 

Website:  N/A 

System  Cost:  To  be  determined  (non-COTS  device) 

Assay  Cost:  $2  per  disk 

Assay  Storage  Requirements:  4°C 

Agents  Tested  per  Assay:  Up  to  20  possible 

Assay  Shelf  Life:  6  months  at  4  °C 

Sample  Size  Required:  2  pL 

Type  of  Detection:  Fluorescent  labeled  antibodies 

Time  to  Result:  20  minutes 

System  Weight:  3.5  lbs  (system  also  requires  laptop) 
Operating  Range:  59-86  °  F  (15  -  30  °  C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  is  early  prototype  unit,  components  are  not 
final.  Assay  development  in  progress. 

Q  £  ifii  7  A 


System  Description 


The  SpinDx  by  Sandia  National  Laboratory  is  a  non- 
COTS  system  designed  primarily  for  rapid  diagnostics 
in  a  clinical  or  point-of-care  setting.  The  system  utilizes 
a  spinning  disk,  or  “lab  on  a  CD”,  to  draw  a  sample 
through  a  separation  matrix  while  simultaneously  binding 
to  fluorescently  labeled  antibody-bead  constructs.  The 
SpinDx  uses  LED  illumination  and  a  photodiode  to  detect 
the  target.  Preliminary  results  from  Sandia  show  the 
system  to  have  greater  detection  sensitivity  than  standard 
ELISAs.  The  SpinDx  has  also  been  used  to  separate  whole 
blood  samples  for  cell  counts  and  other  clinical  analyses. 
Operation  of  the  system  requires  little  or  no  training,  and 
samples  require  no  preparation  except  for  mixing  with 
the  analytical  matrix.  The  SpinDx  is  controlled  by  a  laptop 
computer  via  wireless  (Bluetooth)  communications.  With 
an  analysis  time  of  less  than  20  minutes,  no  sample 
preparation,  battery  power  and  small  size,  the  SpinDx  has 
potential  to  be  a  prototypical  mobile  laboratory  or  even 
hand-held  instrument. 


Test  Bed  Review 


The  results  of  the  laboratory  assessment  indicate  the 
SpinDx  is  still  at  the  developmental  prototype  stage.  Sandia 
utilized  Critical  Reagent  Program  antibodies  to  create 
assays  for  all  5  targets;  however,  the  assays  were  not 
able  to  definitively  detect  any  of  the  targets  in  the  current 
configuration  of  the  SpinDx.  The  analytical  matrix  was 
rather  gel-like,  and  adding  3  uL  of  the  sample-matrix  mix 
to  the  port  on  the  disk  was  difficult  and  not  precise.  Sandia 
supplied  a  specific  pipet  and  plastic  tips  to  load  the  device, 
but  the  sample  loading  step  could  still  be  improved.  The 
operation  of  the  device  was  guided  by  software  on  a  laptop 
computer  connected  wirelessly  to  the  SpinDx.  Because  of 
concerns  about  the  software  measurement  of  analytical 
beads  after  the  “spin”,  Sandia  provided  calibration  beads. 
However,  the  analytical  software  indicated  either  the 
calibration  beads  did  not  run  properly  in  the  disks  or  the 
detection  algorithm  was  errant.  The  battery  re-charger 
connection  interferes  with  closing  of  the  device  lid.  The 
system’s  sample  matrix  loading  and  the  analytical  software 
both  seemed  in  need  of  improvement  for  environmental 
sample  analysis. 

The  SpinDx  was  assessed  by  the  Mobile  laboratory 
operators,  but  not  by  the  field  operators.  Because  of  some 
difficulty  in  mixing  the  sample  with  analytical  matrix  and 
pipeting  into  the  disk  port,  as  well  as  running  the  control 
software,  the  operators  rated  the  “Ease  of  Use”  and  “Data” 
viewing  and  interpreting  attributes  as  being  only  FAIR. 

The  operators  rated  “Training  Simplicity”,  “Safety”  and 
“Maintenance”  all  as  EXCELLENT.  The  operators  also  were 
favorable  to  the  overall  design  of  the  instrument  and  the 
short  musical  tune  that  denotes  completion  of  an  assay. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program. 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

N/A 

O 

N/A 

Yersinia  pestis 

N/A 

O 

N/A 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

N/A 

O 

N/A 

Yersinia  pestis 

N/A 

O 

N/A 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

©  Validated  f%  Not  Validated 

(<1  log  difference) 


Q  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 

100 


_OJ 

01 

in 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 


development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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SPIRIT™ 


by  Seattle  Sensors  Corporation 


System  Specifications 


Vendor:  Seattle  Sensors  Corporation 
Website:  www.seattlesensors.com 

System  Cost:  $35,000.00 
Assay  Cost:  $423.00 

Assay  Storage  Requirements:  Refrigeration 
Agents  Tested  per  Assay:  One 
Assay  Shelf  Life:  6  months  at  4  °C 
Sample  Size  Required:  100-150  p L 
Type  of  Detection:  Antibody 
Time  to  Result:  5-10  minutes 
System  Weight:  3  lbs 
Operating  Range:  59-86  °F  (15-30  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  Software  is  not  yet  final.  System  not  tested  to  full 
capability. 


System  Description 


The  SPIRIT  by  Seattle  Sensors  Systems  Corporation  is  a 
shoe-box  sized  biological  agent  identifier  that  uses  surface 
plasmon  resonance  (SPR)  to  detect  and  measure  binding, 
such  as  between  an  antibody  and  a  specific  bacteria.  The 
SPIRIT  has  condensed  the  research  laboratory  technique 
of  SPR  to  a  portable  device  capable  of  detecting  bacteria, 
viruses  or  toxins  from  complex  samples  by  utilizing  Texas 
Instruments’  Spreeta  SPR  chips.  A  laptop  computer  is  used 
to  control  the  peristaltic  pumps  and  valves  and  to  regulate 
the  flow  of  sample  and  buffer  onto  the  SPR  flowcells  such 
that  each  sample  may  be  analyzed  within  25  minutes. 

The  operator  monitors  the  SPR  signals  through  a  graphic 
display  and  post-run  data  analysis.  The  system  allows  for 
regeneration  of  the  Spreeta  chip  surfaces,  such  that  up 
to  100  samples  may  be  analyzed  before  the  chip  must  be 
replaced.  Among  the  mobile  and  man-portable  detection 
systems,  SPIRIT  has  relatively  fast  assay  times,  a  small 
footprint  and  battery-power. 


Test  Bed  Review 


Seattle  Sensors  Systems  developed  assays  for  all  5  test 
agents  through  binding  antibodies  from  the  Critical  Reagent 
Program  to  individual  SPR  chips.  The  software  version 
included  with  the  SPIRIT  did  not  allow  full  functionality; 
consequently,  only  singleplex  detection  capability  was 
assessed  on  this  version  of  the  SPIRIT.  The  laboratory 
assessment  utilized  individual  targets  on  channel  4  and 
a  calibration  control  on  channel  3,  while  channels  1  and 
2  were  not  configured  for  data  collection  in  this  software 
release.  The  set-up,  priming  and  calibration  of  the  SPIRIT 
took  about  30  minutes,  while  sample  data  collection  took 
approximately  25  minutes.  The  SPIRIT  was  at  the  lower 
end  of  sensitivity  among  the  biological  agent  identifiers 
as  bacteria  samples  were  detected  at  107  CFU/mL  and 
Vaccinia  virus  inconsistently  identified  at  107  PFU/mL. 
Meanwhile,  VEE  was  not  detected  at  108  PFU/mL  or  BoNT  A 
at  1  ug/mL.  The  SPIRIT  required  some  training  and  practice 
for  the  operators  to  become  comfortable  with  performing 
assays. 

The  SPIRIT  received  generally  favorable  usability  scores 
from  Mobile  laboratory  operators,  with  an  EXCELLENT  and 
FAIR  rating  for  “Ease  of  Use”  and  GOOD  or  EXCELLENT 
scores  for  Data,  Training,  Safety  and  Maintenance 
categories.  The  operators  had  some  troubles  with 
the  injection  of  sample  to  the  port  and  experienced 
inconsistent  internal  peristaltic  pump  pressures.  The  SPIRIT 
was  sensitive  to  work  surface  vibrations  perturbing  the 
data  collection.  Of  importance  to  sample  size  requirements, 
one  operator  was  concerned  that  “1  mL  of  sample  was 
needed”  which  may  use  too  much  of  their  collected  sample. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Laboratory  Usability  Scores 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


JlNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

No  Claim 

O 

>1.00xl07 

Yersinia  pestis 

No  Claim 

O 

l.OOxlO7 

Vaccinia 

No  Claim 

O 

1.00x10s 

Venezuelan  Equine 
Encephalitis 

No  Claim 

O 

>1.00xl08 

Clostridium 

Botulinum  Toxin 

No  Claim 

O 

>1.00x10® 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

>1.00xl07 

O 

N/A 

Yersinia  pestis 

l.OOxlO7 

O 

N/A 

Vaccinia 

1.00x10s 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

>1.00xl08 

O 

N/A 

Clostridium 

Botulinum  Toxin 

>1.00xl03 

O 

N/A 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


Q  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 

vi  i  T  3 


MOBILE 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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NUCLEIC-ACID  TECHNOLOGIES 


FilmArray® 


by  BioFire™  Diagnostics 


System  Specifications 


Vendor:  BioFire  Diagnostics,  Inc. 

Website:  www.biofiredx.com 

System  Cost:  $49,500.00 
Assay  Cost:  $185 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  17  (BioThreat  Panel)  including  multiple 
targets  per  agent 

Assay  Shelf  Life:  4-6  months  at  room  temperature 

Sample  Size  Required:  250  pL 

Type  of  Detection:  Nucleic  acid  amplification  with  end  point 
melt-curve  analysis 

Time  to  Result:  65  minutes 

System  Weight:  20  lbs  (systems  requires  laptop  and  pouch 
preparation  station  not  included  in  weight) 

Operating  Range:  59-86  °  F  (15  -  30  °  C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab. 


System  Description 


The  BioFire  FilmArray  system  is  a  multiplex  PCR  system 
that  integrates  sample  preparation,  amplification,  detection 
and  analysis.  The  BioThreat  Panel  v2.4  pouch  stores  all 
the  necessary  reagents  for  sample  preparation,  RT- 
PCR,  PCR  and  detection  in  a  freeze-dried  format.  Once 
an  unprocessed  sample  is  injected  in  to  the  pouch  the 
FilmArray  will  extract  and  purify  nucleic  acids;  perform  an 
initial,  large  volume  multiplex  PCR;  and  complete  individual 
singleplex,  second-stage  PCR  reactions  to  identify  specific 
targets.  Finally  the  system  uses  meta  analysis  of  endpoint 
melt  curve  data  to  generate  a  result  for  each  agent  based 
on  the  results  of  one  or  more  targets,  each  performed 
in  triplicate.  Each  assay  contains  internal  standards  to 
automatically  control  for  each  step  of  the  process.  The 
design  of  this  system  requires  minimal  user  training  and 
very  little  hands  on  time. 


Test  Bed  Review 


The  FilmArrays’  ability  to  screen  17  different  pathogens 
(25  targets  total)  in  60  minutes  with  sensitivity  as  low 
as  5xl02  CFU/ml  makes  this  system  an  asset  in  any 
laboratory.  Although  the  FilmArray  is  not  “portable”,  the 
unique  BioThreat  pouch  had  the  ability,  during  our  study, 
to  simultaneously  detect  3  targets  of  Bacillus  anthracis , 

2  targets  of  Yersinia  pestis,  2  targets  of  VEE  virus  and 
2  targets  of  Orthopox  virus.  During  testing  the  targets 
were  run  as  singleplex  samples,  and  then  the  system  was 
challenged  using  multiple  targets  in  a  single  sample.  The 
FilmArray  was  able  to  detect  all  4  targets  with  no  loss  to 
sensitivity. 

During  the  mobile  laboratory,  scientists  were  impressed 
with  the  ease  of  use  that  the  pouch  and  its  associated 
stand  offered.  A  quick  10  minute  training  with  each 
end-user  allowed  them  to  run  the  system  independently 
with  one  scientist  commenting,  “I  wish  every  system  was 
this  easy  to  run”.  The  biggest  impact  the  system  had  on 
the  Mobile  Lab  scientists  was  its  multiplex  capability. 

“For  screening  an  unknown  sample,  this  system  would 
be  invaluable”,  one  commented.  With  time  not  being  an 
immediate  concern  for  this  group,  the  FilmArray  was  among 
their  top  picks  of  systems. 
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NUCLEIC-ACID  TECHNOLOGIES 


Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

5.00xl02 

O 

5.00x10s 

Yersinia  pestis 

5-OOxlO1 

© 

5.00x10° 

Vaccinia 

l.OOxlO2 

O 

l.OOxlO3 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

O 

l.OOxlO6 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

5.00x10s 

© 

5.00x10s 

Yersinia  pestis 

5.00x10° 

O 

5-OOxlO1 

Vaccinia 

l.OOxlO3 

© 

l.OOxlO3 

Venezuelan  Equine 
Encephalitis 

l.OOxlO6 

© 

l.OOxlO6 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Laboratory  Usability  Scores 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 

(0^2 - 3 - 4 - S-l 


MOBILE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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NUCLEIC-ACID  TECHNOLOGIES 


B  I  O 


FIRE 


Overview 

The  BioFire  FilmArray  system  is  a  multiplex  PCR  system  that 
integrates  sample  preparation,  amplification,  detection,  and 
analysis.  The  BioThreat  Panel  v2.4  specific  pouch  stores  all 
the  necessary  reagents  for  sample  preparation,  real  time- 
PCR  (RT-PCR),  PCR,  and  detection  in  a  freeze-dried  format. 

Once  an  unprocessed  sample  is  injected  in  to  the  pouch,  the 
FilmArray  will  extract  and  purify  nucleic  acids;  perform  an 
initial,  large  volume  multiplex  PCR;  and  complete  individual 
singleplex,  second-stage  PCR  reactions  to  identify  specific 
targets.  Finally  the  system  uses  meta-analysis  of  endpoint  melt  curve  data  to  generate  a  result  for  each 
agent  based  on  the  results  of  one  or  more  targets,  each  performed  in  triplicate.  Each  assay  contains 
internal  standards  to  automatically  control  for  each  step  of  the  process.  The  design  of  this  system 
requires  minimal  user  training  and  very  little  hands  on  time. 


The  BioThreat  Panel  v2.4  is  a  semi-automated  assay  format  for  which  the  user  mixes  the  sample  into  a 
dilution  matrix.  The  sample  mix  and  a  separate  rehydration  buffer  are  drawn  into  syringes  and  injected 
into  matching  color-coded  ports  on  the  assay  cassette.  Sample  preparation,  injection  of  the  cassette, 
and  software  setup  is  completed  in  approximately  two  minutes.  The  run-time  of  the  BioThreat  Panel 
v2.4,  including  automated  data  analysis,  is  65  minutes.  BioFire  currently  has  two  panels  that  are  FDA- 
cleared  (Respiratory  Panel,  Blood  Culture  Identification  Panel);  however,  for  this  evaluation  the  newer 
BioThreat  panel,  designed  to  detect  17  pathogens  and  25  different  targets,  was  used. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Gamma-irradiated  Bacillus  anthracis  spores  were  diluted  to  5X101,  5xl02,  and  5xl03  Colony  Forming 
Units  (CFU)/mL  in  Phosphate  Buffered  Saline  (PBS)  solution  then  tested,  in  triplicate,  using  the  BioThreat 
Panel  v2.4  assay  pouch.  Approximately  210  microliters  (pL)  of  sample  was  transferred  to  the  sample 
buffer  and  injected  into  the  pouch  which  then  draws  in  a  pre-determined  aliquot  of  the  sample/sample 
buffer  mixture  by  vacuum.  Although  the  vendor  stated  the  LOD  to  be  5xl02  CFU/mL,  consistent 
detection  required  a  10-fold  increase  to  5xl03  CFU/mL. 


Table  6.  Bacillus  anthracis  LOD 


Concentration 
(Colony 
Forming  Units 
(CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents 
(GE) 

Results  by  Loci 
(Positives/Total  Runs) 

Result  by 
Agent 

Chromosome 

pXOl 

pX02 

B.  anthracis 

5.00xl03 

1.05xl03 

2.49xl03 

3/3 

3/3 

3/3 

3/3 

5.00xl02’ 

1.05xl02 

2.49xl02 

1/3 

0/3 

2/3 

0/3 

5.00X101 

1.05X101 

2.49X101 

0/3 

0/3 

0/3 

0/3 

Vendor  Claimed  LOD:  5.00xl02  CFU/mL 
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NUCLEIC-ACID  TECHNOLOGIES 


B  I  O 


FIRE 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma-irradiated  Yersinia  pestis  cells  were  diluted  to  5x10°,  5X101,  and  5xl02  CFU/mL  in  PBS  then 
tested,  in  triplicate,  using  the  BioThreat  Panel  v2.4  assay  pouch.  Approximately  210pL  of  sample  was 
transferred  to  the  sample  buffer  and  injected  in  to  the  loading  dock  which  then  draws  in  a  pre¬ 
determined  aliquot  of  the  sample  mixture  by  vacuum.  The  vendor  claimed  LOD  of  this  assay  was  5X101 
CFU/mL,  however  testing  5  CFU/mL  was  successfully  detected. 


Table  7.  Yersinia  pestis  LOD 


Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

Result  by 
Agent 

pPCPl 

pMTl 

Y.  pestis 

5.00xl02 

1.05xl02 

1.45xl02 

3/3 

3/3 

3/3 

5.00X101' 

1.05X101 

1.45X101 

3/3 

1/3 

3/3 

5.00x10° 

1.05x10° 

1.45x10° 

3/3 

0/3 

3/3 

*Vendor  Claimed  LOD:  5-OOxlO1  CFU/mL 


Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  virus  was  diluted  to  lxlO1,  lxlO2,  and  1x10s  PFU/mL  in  PBS  then  tested,  in 
triplicate,  using  the  BioThreat  Panel  v2.4  assay  pouch.  The  sample  volume  transferred  to  the  Sample 
Buffer  was  measured  to  be  approximately  210pL.  The  system  draws  in  a  pre-determined  aliquot  of  the 
sample  mixture  by  vacuum.  The  vendor  stated  LOD  of  this  assay  is  lxlO2  PFU/mL,  however  10-fold  more 
sample,  lxlO3  PFU/mL,  was  required  for  consistent  detection  in  this  study. 


Table  8.Vaccinia  LOD 


Concentration 

Results  by  Loci 

Result  by 

(Plaque  Forming 

Total  PFU 

Total  GE 

(Positives/Total  Runs) 

Agent 

Units  (PFU)/mL) 

OPX2 

VAR3 

VAC 

l.OOxlO3 

2.09xl02 

3.02xl02 

3/3 

3/3 

3/3 

l.OOxlO2* 

2.09X101 

3.02X101 

1/3 

1/3 

1/3 

l.OOxlO1 

2.09x10° 

3.02x10° 

0/3 

0/3 

0/3 

*Vendor  Claimed  LOD:  l.OOxlO2  PFU/mL 
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NUCLEIC-ACID  TECHNOLOGIES 


B  I  O 


FIRE 


Venezuelan  Equine  Encephalitis 

+  sense  single-stranded  ribonucleic  acid  (ssRNA),  Alphavirus 

Gamma-irradiated  VEE  virus  was  diluted  to  lxlO2,  lxlO3,  lxlO4,  lxlO5,  and  lxlO6  PFU/mL  in  PBS  then 
tested,  in  triplicate,  using  the  BioThreat  Panel  v2.4  assay  pouch.  The  sample  volume  transferred  to  the 
Sample  Buffer  was  measured  to  be  approximately  210pL.  The  system  draws  in  a  pre-determined  aliquot 
of  the  sample  mixture  by  vacuum.  The  vendor  claimed  LOD  of  this  assay  was  lxlO3  PFU/mL,  however 
1000-fold  more  sample,  lxlO6  PFU/mL  than  the  lxlO3  PFU/mL  LOD  described  by  the  vendor. 


Table  9.  Venezuelan  Equine  Encephalitis  LOD 


Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

Result  by 
Agent 

MP2 

RC3 

VEE 

1.00x10s 

2.09xl05 

2.09xl05 

3/3 

3/3 

3/3 

1.00x10s 

2.09xl04 

2.09xl04 

2/3 

1/3 

2/3 

l.OOxlO4 

2.09xl03 

2.09xl03 

1/3 

0/3 

1/3 

l.OOxlO3* 

2.09xl02 

2.09xl02 

0/3 

0/3 

0/3 

l.OOxlO2 

2.09X101 

2.09X101 

0/1 

0/1 

0/1 

*Vendor  Claimed  LOD:  l.OOxlO3  PFU/mL 


Multiplex  Testing 

Based  on  the  results  of  individual  singleplex  target  detection  assessments,  samples  were  prepared 
containing  agents  at  the  following  concentrations:  Bacillus  anthracis  at  5.00xl03  CFU/mL,  Yersinia  pestis 
at  5.00x10°  CFU/mL,  VAC  at  1.00x10s  PFU/mL,  and  VEE  at  l.OOxlO6  PFU/mL.  All  four  targets  were 
detected  in  all  three  samples  with  the  exception  of  Yersinia  pestis ,  which  was  only  detected  in  two  of 
three  samples.  Fresh  samples  were  prepared  with  the  concentration  of  Yersinia  pestis  increased  10-fold 
while  all  other  target  concentrations  remained  the  same.  In  the  repeat  testing,  all  three  samples  were 
reported  positive  for  all  four  agents. 


Table  10.  Multiplex  Evaluation 


Agent 

Concentrati 
on  (CFU  or 
PFU/mL) 

Total  CFU 

or  PFU 

B.  anthracis 

5.00xl03 

1.05xl03 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

Result  by  Agent 

Chrom 

pXOl 

pX02 

B.  anthracis 

2.49xl03 

6/6 

6/6 

6/6 

6/6 

pPCPl 


Y.  pestis 


5.00x10° 

1.05x10° 

1.45x10° 

2/3 

0/3 

2/3 

5-OOxlO1 

1.05X101 

1.45X101 

3/3 

3/3 

3/3 

OPX2 

VAR  3 

Orthopox 

l.OOxlO3 

2.09xl02 

3.02xl02 

5/6 

3/6 

6/6 

MP2 

RC3 

VEE 

l.OOxlO6 

2.09x10s 

2.09x10s 

6/6 

6/6 

6/6 

Discussion 


Call  Assignments 

One  concern  observed  with  the  system  was  the  arbitrary  manner  in  which  positive  and  negative  system 
calls  of  pathogen  identifications  are  made.  In  most  cases,  a  positive  call  was  made  if  any  single  locus  was 
detected.  Anecdotal  experience  with  large  surveillance  programs  such  as  the  Department  of  Homeland 
Security's  (DHS)  BioWatch  and  JPEO-CBD  efforts  indicates  that  single  locus  detection  of  bacterial 
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pathogens  has  an  unacceptable  high  false  positive  rate,  especially  in  environmental  samples.  This  is 
particularly  true  when  the  definitive  locus  is  present  on  plasmids,  which  are  often  shared  between 
bacteria  via  horizontal  gene  transfer.  As  a  result,  ambiguous  results  are  sometimes  generated  but  not 
reported  as  "ambiguous".  For  example,  a  low-level  detection  event  identified  by  the  single  most 
sensitive  locus  PCR  reaction  could  be  confused  with  a  singleplex  positive  result  from  a  genetic  near¬ 
neighbor  sharing  a  genetic  target  with  the  organism  of  interest. 


This  approach  also  discards  one  of  the  most  desirable  features  of  multiplex  detection:  the  power  of  joint 
probability  distributions  of  multiple  positive,  independently  developed  assays  to  provide  higher 
confidence  results  than  possible  with  any  single  assay  alone.  For  example,  three  independent  assays  for 
a  given  pathogen  might  have  a  false  positive  rate  of  10‘2,  but  the  simultaneous  detection  of  all  three  loci 
would  result  in  a  calculated  false  positive  rate  for  the  organism  of  10  6. 


It  is  worth  noting  that  broad  and  inclusive  detection  of  highly  divergent  organisms,  such  as  ssRNA 
viruses,  may  require  multiple  loci  to  capture  the  entire  detection  space.  For  example,  one  locus  may 
detect  one  serotype,  while  another  may  be  required  to  detect  other  serotypes  of  the  same  species.  In 
this  case,  each  locus  would  be  considered  as  a  subset  of  a  single  assay  for  simple  terms  of  inclusivity  and 
any  positive  result  of  this  assay  would  be  considered  a  positive  result. 

Plasmid  and  Chromosomal  Targets 

The  utility  of  using  both  plasmid  and  chromosomal  PCR  targets  has  the  advantage,  as  noted  above,  of 
high  confidence  identification  of  the  organism  and  the  ability  to  look  for  multiple  virulence  determinants 
found  on  separate  elements  of  the  genomic  architecture.  Because  Bacillus  plasmids  may  transfer 
between  species  without  conference  of  virulence,  reliance  on  plasmids  alone  as  PCR  targets  may  result 
in  higher  false  positive  rates.  In  the  BioThreat  Panel  v2.4,  the  B.  anthracis  assay  includes  what  has 
become  a  standard  configuration:  a  chromosomal  target  and  one  target  for  each  of  the  two  virulence 
plasmids  (pXOl  and  pX02),  allowing  for  simultaneous  high  confidence  detection  of  the  organism  and 
the  known  pathogenicity  islands.  However,  the  Y.  pestis  assay  has  only  two  of  the  three  virulence 
plasmids,  and  omits  any  chromosomal  targets.  The  addition  of  targets  on  the  third  virulence  plasmid 
(pCDl)  and  the  chromosome  would  be  of  significant  advantage  for  a  more  complete  and  actionable 
assay. 


One  distinct  benefit  of  targeting  plasmids  is  the  fact  that  they  are  usually  multi-  or  high-copy  replicons, 
which  increases  the  sensitivity  of  these  assays  for  a  given  CFU  of  input  material.  Unfortunately,  the 
sensitivity  of  the  various  BioThreat  Panel  v2.4  PCR  assays  does  not  seem  to  reflect  the  expected  increase 
in  sensitivity  for  plasmid  targets  as  expected.  In  the  Bacillus  anthracis  assay,  there  is  a  suggestion  that 
the  pX02  assay  is  indeed  slightly  more  sensitive  than  the  chromosomal  target,  but  additional  testing 
with  finer  graduations  of  template  concentrations  and  more  replicates  would  be  required  to  confirm  this 
preliminary  finding.  However,  even  with  this  limited  data,  it  is  clear  that  the  pXOl  assay  is  approximately 
10-fold  less  sensitive  that  the  other  two  assays.  The  disparity  in  expected  PCR  performance  (more  than 
an  order  of  magnitude  without  considering  the  differences  in  template  concentrations  between  the 
chromosome  and  plasmid)  raises  some  concern  that  the  optimization  of  the  individual  PCR  reactions 
requires  some  additional  work. 

Assay  Sensitivity 

The  laboratory  determined  assay  LOD  for  VEE  is  approximately  three  orders  of  magnitude  higher  than 
that  stated  in  the  product  literature.  The  two  bacterial  assays  demonstrated  sensitivity  concordant  with 
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vendor  claims,  and  the  dsDNA  virus  target  (VAC)  did  not  have  a  published  LOD.  It  is  possible  that  the  lack 
of  sensitivity  of  the  VEE  assay  is  related  to  either  variability  of  the  input  template  (viral  RNA  preparations 
can  vary  significantly  as  a  function  of  total  target  RNA  to  PFU  ratios  in  different  production  lots)  or  the 
efficiency  of  the  RNA  purification  and  reverse  transcription  to  complementary  DNA  (cDNA).  Additional 
investigation  to  determine  the  sensitivity  of  cDNA  versus  native  RNA  targets  and  more  information  from 
the  assay  developers  may  shed  light  on  the  disparity  observed  in  assay  performance. 


The  observed  sensitivity  of  the  VAC  assay  was  moderately  less  in  multiplex  versus  the  singleplex  format. 
All  other  assays  seemed  to  be  equivalently  sensitive  in  either  format.  One  of  the  assays  (VAR3)  was 
particularly  affected  (only  50%  detection  rate  at  the  LOD  in  multiplex  versus  100%  in  singleplex).  This 
phenomenon  is  not  unexpected  because  of  known  issues  with  competition  for  reagents  that  require 
precise  balancing  of  simultaneous  reactions  in  multiplex  PCR.  However,  some  additional  multiplex  PCR 
optimization,  and  the  potential  need  to  screen  additional  pan-orthopox  assays,  will  be  required  to 
demonstrate  the  same  level  of  even  performance  shown  with  the  other  organisms. 
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RAZOR® 


by  BioFire  Diagnostics,  Inc. 


System  Specifications 


Vendor:  BioFire  Diagnostics,  Inc. 

Website:  www.biofiredx.com 

System  Cost:  $38,500.00 
Assay  Cost:  $200.00 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  10 

Assay  Shelf  Life:  1  year  at  room  temperature 

Sample  Size  Required:  250  pL 

Type  of  Detection:  Nucleic  Acid 

Time  to  Result:  25  minutes 

System  Weight:  11  lbs 

Operating  Range:  32-104  °F(0-40  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field. 

°  d 


System  Description 


BioFire’s  RAZOR  EX  is  a  field  PCR  unit  that  uses  pouches 
pre-loaded  with  freeze-dried  PCR  reagents  for  the  detection 
and  identification  of  biological  pathogens  and  biothreat 
agents.  Each  kit  contains  all  of  the  items  necessary  for 
sampling,  sample  preparation  and  real-time  PCR.  Each 
kit  includes  items  needed  for  collecting  and  loading  the 
sample.  Once  samples  are  loaded  into  the  pouch  with 
cannula-tipped  syringes  they  are  dispensed  automatically 
in  to  the  wells,  requiring  no  precise  measuring.  BioFire 
has  assays  available  for  CDC  defined  Category  A  and 
B  Biothreat  pathogens.  One  pouch  will  test  for  3  to  10 
different  agents,  depending  on  configuration,  and  includes 
internal  controls  to  validate  the  integrity  of  the  test. 
BioFire’s  pouches  and  its  associated  kit  components  are 
manufactured  under  a  cGMP  quality  system.  This  testing 
used  The  TEN™  10  Target  Screen  Kit  to  detect  2  out  of 
4  desired  targets,  Bacillus  anthracis  and  Yersinia  pestis, 
and  verify  vendor  claims  of  limits  of  detection  (LOD). 
Additionally,  the  Botulinum  toxin  preparation  was  tested  for 
residual  C.  botulinum  DNA. 


Test  Bed  Review 


The  RAZOR  was  one  of  the  few  systems  in  this  test  that  had 
been  designed  as  a  field-ready  system.  Because  of  this,  the 
expectation  that  it  would  outperform  the  others  was  there, 
however  the  ease  of  use  did  not  meet  up  with  the  end 
users  requirements. 

In  the  laboratory  the  system  performed  comparable  to  the 
other  PCR  systems,  with  LOD’s  down  to  102,  for  its  Yersinia 
pestis  assay.  Unfortunately,  BioFire’s  RAZOR  is  considered 
a  “closed”  system,  the  company  was  not  willing  to  develop 
new  assays  for  this  study  therefore  only  2  ( Yersinia  pestis 
and  Bacillus  anthracis)  of  the  desired  5  targets  were 
tested.  This  closed  system  is  an  unfavorable  characteristic 
for  military  applications  that  are  challenged  with  new 
threats  and  require  new  assays  at  any  given  time. 

During  the  field  testing,  soldiers  had  difficulty  manipulating 
the  pouch  and  found  the  barcode  scanner  difficult  to 
use.  One  commented,  “This  system  is  tedious”.  Another 
concern  for  the  soldiers  in  the  field  was  the  difficulty  they 
had  reading  the  screen  in  the  daylight.  One  soldier  noted 
“the  brightness  option  does  not  help  in  the  bright  sunlight”. 
Overall  the  field  ready  system  isn’t  as  ready  as  it  appears. 

In  full  MOPP  gear  the  consumables  gave  the  greatest 
challenge  even  for  the  simplest  of  tasks,  such  as  opening 
a  box. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

l.OOxlO3 

1.30xl04 

Yersinia  pestis 

l.OOxlO2 

O 

1.30xl03 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

1.30xl04 

© 

1.30xl04 

Yersinia  pestis 

1.30xl03 

© 

1.30xl03 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Laboratory  Usability  Scores 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


Q  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 

v  c  i  7  3 

MOBILE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 

BioFire's  RAZOR  EX  is  a  field  PCR  unit  that  uses  pouches  pre- 
loaded  with  freeze-dried  PCR  reagents  for  the  detection  and 
identification  of  biological  pathogens  and  biothreat  agents. 

Each  kit  contains  all  of  the  items  necessary  for  sampling, 
sample  preparation,  and  real-time  PCR.  Each  kit  includes 
items  needed  for  collecting  and  loading  the  sample.  Once 
samples  are  loaded  into  the  pouch  with  cannula-tipped 
syringes  they  are  dispensed  automatically  into  the  wells, 
requiring  no  precise  measuring.  BioFire  has  assays  available 
for  CDC-defined  Category  A  and  B  Biothreat  pathogens.  One 
pouch  will  test  for  three  to  10  different  agents,  depending  on 
configuration,  and  includes  internal  controls  to  validate  the 
integrity  of  the  test.  BioFire's  pouches  and  its  associated  kit 
components  are  manufactured  under  a  Current  Good 

Manufacturing  Practices  (cGMP)  quality  system.  This  testing  used  The  TEN®  Target  Screen  Kit  to  detect 
two  out  of  four  desired  targets,  Bacillus  anthracis  and  Yersinia  pestis,  and  verify  vendor  claims  of  LOD. 
Additionally,  the  Botulinum  toxin  preparation  was  tested  for  residual  C.  botulinum  DNA. 


The  RAZOR  EX  contains  a  barcode  reader  on  the  rear  of  the  device.  A  user  selects  the  option  to  run  an 
assay  from  the  device  home  screen  then  is  prompted  to  scan  the  barcode  of  the  assay.  Once  the  assay  is 
recognized  by  the  instrument  the  user  is  prompted  to  prepare  the  sample  and  load  it  into  the  kit 
through  onscreen  step  by  step  instructions.  First,  the  user  draws  an  aliquot  of  a  negative  control 
(provided  in  each  kit)  into  a  syringe  and  pushes  the  plastic  cannula  into  the  marked  port  of  the  pouch. 
Vacuum  draws  the  required  amount  of  liquid  into  the  wells  to  rehydrate  negative  and  inhibition 
controls.  Second,  the  user  mixes  the  sample  into  5mLs  of  sample  diluents  (also  provided  in  each  kit  and 
labeled  as  "unknown"),  draws  a  portion  of  the  sample  mixture  into  a  syringe,  and  inserts  the  plastic 
cannula  into  the  marked  port.  Again,  vacuum  draws  the  requisite  amount  into  the  pouch.  Lastly,  the 
user  removes  a  comb  from  around  the  plungers,  rotates  the  plungers,  and  pushes  them  down  to  force 
the  rehydrated  reagents  into  the  assay  strips  which  contain  the  primer  and  probe  oligonucleotides. 
Preparing  the  sample,  injecting  the  pouch,  and  twisting  the  plungers  requires  approximately  five 
minutes.  Once  the  properly  prepared  pouch  is  installed  onto  the  device,  results  for  all  ten  targets  are 
available  within  25  minutes. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 


Gamma-irradiated  Bacillus  anthracis  spores  were  serially  diluted  in  10-fold  increments  into  water  then 
tested,  in  triplicate,  using  The  TEN®  Target  Screen  Kit.  The  sample  volume  was  transferred  to  the 
"unknown"  bottle  using  the  transfer  pipet  included  in  the  kit.  The  instructions  and  kit  information  list 
the  volume  transferred  to  be  0.5mL  although  it  was  determined  to  be  approximately  265pL  based  on 
the  weight  of  water  transferred.  Therefore,  the  sample  was  further  diluted  nearly  20-fold  before  being 
loaded  into  the  pouch.  The  requisite  amounts  of  water  for  negative  and  inhibition  controls  as  well  as  of 
the  sample  itself  were  drawn  into  the  pouch  by  vacuum.  Although  the  vendor  stated  LOD  of  the  assay 
for  B.  anthracis  is  lxlO3  CFU/mL,  10-fold  more  was  required  for  consistent  detection  of  B.  anthracis  in 
this  study. 
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Table  11.  Bacillus  anthracis  LOD 


Concentration 
(Colony 
Forming  Units 
(CFU)/mL) 

Total  CFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

pX02 

1.30xl05 

3.45xl04 

8.20xl04 

3/3 

1.30xl04 

3.45x10s 

8.20x10s 

3/3 

1.30x10s* 

3.45xl02 

8.20xl02 

2/3 

*Vendor  Claimed  LOD:  1.00x10s  CFU/mL 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma-irradiated  Yersinia  pestis  cells  were  serially  diluted  in  10-fold  increments  into  water  then  tested, 
in  triplicate,  using  The  TEN®  Target  Screen  Kit.  The  sample  volume  was  transferred  to  the  "unknown" 
bottle  using  the  transfer  pipet  included  in  the  kit.  The  instructions  and  kit  information  list  the  volume 
transferred  to  be  0.5mL  although  it  was  determined  to  be  approximately  265pL  based  on  the  weight  of 
water  transferred.  Therefore,  the  sample  was  further  diluted  nearly  20-fold  before  being  loaded  into  the 
pouch.  The  requisite  amounts  of  water  for  negative  and  inhibition  controls  as  well  as  of  the  sample  itself 
were  drawn  into  the  pouch  by  vacuum.  Although  the  vendor  stated  LOD  of  the  assay  for  Y.  pestis  is 
lxlO2  CFU/mL,  10-fold  more  was  required  for  consistent  detection  in  this  study. 

Table  12.  Yersinia  pestis  LOD _ 


Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

pPCPl 

1.30xl04 

3.45x10s 

4.77x10s 

3/3 

1.30x10s 

3.45xl02 

4.77xl02 

3/3 

1.30xl02* 

3.45X101 

4.77X101 

2/3 

*Vendor  Claimed  LOD:  l.OOxlO2 CFU/mL 
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Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  virions  were  serially  diluted  in  10-fold  increments  into  water  then  tested,  in 
triplicate,  using  The  TEN®  Target  Screen  Kit.  The  sample  volume  was  transferred  to  the  "unknown" 
bottle  using  the  transfer  pipet  included  in  the  kit.  The  instructions  and  kit  information  list  the  volume 
transferred  to  be  0.5mL  although  it  was  determined  to  be  approximately  265pL  based  on  the  weight  of 
water  transferred.  Therefore,  the  sample  was  further  diluted  nearly  20-fold  before  being  loaded  into  the 
pouch.  The  requisite  amounts  of  water  for  negative  and  inhibition  controls  as  well  as  of  the  sample  itself 
were  drawn  into  the  pouch  by  vacuum.  Upon  obtaining  all  negative  detections  for  the  1.30xl05  PFU/mL 
sample,  100  times  more  sample  was  tested.  This  sample  was  also  negative.  The  vendor  noted  that  the 
assay  had  been  modified  from  a  pan-  orthopox  assay  to  a  Variola-specific  assay. 

Table  13.  Vaccinia  LOD 


Concentration 
(Plaque  Forming 
Units  (PFU)/mL) 

Total  PFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

VAC 

1.30xl07 

3.45xl06 

4.97x10s 

0/1 

1.30xl05 

3.45xl04 

4.97xl04 

0/3 

*Vendor  Claimed  LOD:  N/A 


Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

No  Assay  Available 

Clostridium  botulinum  Type  A  Toxin 
Protein  toxin 


The  TEN®  Target  Screen  Kit  includes  an  assay  for  Clostridium  botulinum,  although  the  target  is  DNA 
rather  than  the  active  toxin  used  in  this  assessment.  The  BoNT  A,  as  supplied  by  Metabiologics,  is 
produced  in  its  native  organism;  therefore,  the  agent  was  tested  to  determine  whether  the  system  could 
detect  residual  C.  botulinum  DNA  in  the  toxin  preparation.  The  device  did  not  detect  residual  DNA, 
which  is  consistent  with  information  obtained  from  Metabiologics  that  the  products  have  very  low  A26o 
measurements. 

Table  14.  BoNT  A  LOD 


Concentration 
(Nanograms  (ng)/mL) 

Total  Nanograms 

Results  by  Loci 
(Positives/Total  Runs) 

Unknown 

l.OOxlO3 

250 

0/1 

l.OOxlO2 

25 

0/3 

*Vendor  Claimed  LOD:  N/A 


Multiplex  Detection 

Based  on  the  results  of  individual  singleplex  target  detection  assessments,  separate  samples  were 
prepared  containing  B.  anthracis  at  2.60xl04  CFU/mL  and  Y.  pestis  at  2.00xl03  CFU/mL.  The  two  samples 
were  combined  in  equal  portions  to  create  the  test  sample,  which  was  tested  in  triplicate.  Both  B. 
anthracis  and  Y.  pestis  were  detected  in  all  three  samples. 
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Table  15.  Multiplex  Evaluation 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

B.  anthracis 

1.30xl04 

3.44x10s 

8.19x10s 

3/3 

Y.  pestis 

l.OOxlO3 

2.65xl02 

3.68xl02 

3/3 

Discussion 
Call  Assignments 

Instrument  calls  are  assigned  by  single  locus  amplification  and  real-time,  exponential  single  color 
fluorescence  measurements.  A  single  sample  can  be  screened  against  up  to  ten  loci/agents  in 
approximately  30  minutes  including  integrated  sample  preparation.  Aliquots  of  the  purified  templates 
are  diverted  into  individual  channels  for  separate  single-color  real-time  PCR  reactions,  eliminating  issues 
of  PCR  reaction  interference  and/or  color  separation  issues  that  might  provide  lower  sensitivity  and 
specificity.  However,  anecdotal  experience  with  large  surveillance  programs  such  as  DHS'  BioWatch  and 
JPEO-CBD  would  indicate  that  single  locus  detection  of  bacterial  pathogens  has  an  unacceptable  high 
false  positive  rate,  especially  in  environmental  samples.  As  configured,  the  system  is  appropriate  for 
low-throughput  applications  such  as  screening  suspicious  bulk  materials/high  titer  clinical  samples  for 
presumptive  presence  of  a  threat  agent.  In  circumstances  with  a  known  and  verified  threat  (such  as  an 
existing  outbreak),  this  would  be  sufficient  for  a  positive  assignment.  For  routine  surveillance,  multi¬ 
locus  and/or  orthogonal  confirmation  with  culture  or  immunoassay  would  be  required  for  verification. 

Plasmid  and  Chromosomal  Targets 

Both  the  B.  anthracis  (pX02)  and  Y.  pestis  (pPCPl)  assays  identify  plasmid  targets.  For  singleplex  assays, 
this  has  both  advantages  and  disadvantages.  Sensitivity  of  plasmid  targets  (especially  smaller  and 
potentially  higher-copy  replicons  such  as  pPCPl)  is  usually  superior  to  that  achievable  with 
chromosomal  targets.  However,  the  known  issues  of  horizontal  gene  transfer  and  plasmid  sharing  often 
seen  between  closely  related  near  neighbor  species  can  cause  problems  with  specificity  and  generate 
false-positive  reactions  and  instrument  calls.  Due  to  the  limited  testing  (only  two  bacteria  and  a  single 
viral  locus  with  no  appropriate  template  control  available)  and  lack  of  additional  assay  targeted  at  other 
discrete  loci,  there  is  no  way  to  evaluate  specificity  versus  sensitivity  trade-offs  in  this  study. 

The  choice  to  include  only  a  Variola-specific  assay  is  problematic  for  two  reasons.  The  vendor  did  not 
provide  the  positive  control  (plasmid  containing  expected  amplicon)  and  federal  regulations  do  not 
permit  the  distribution  of  the  Variola  genome,  which  did  not  allow  for  laboratory  testing  to  verify 
vendor  claims.  This  unfortunately  also  does  not  allow  for  simultaneous  detection  of  other  known 
dangerous  pathogens  in  the  orthopox  family  (Monkeypox,  Camelpox,  and  Vaccinia).  Inclusion  of  a  pan¬ 
orthopox  assay  would  have  given  a  broader  representation  of  poxvirus  pathogen  detection,  as  well  as 
allowing  for  routine  testing  with  unregulated  materials. 

Assay  Sensitivity 

For  a  PCR  platform,  the  sensitivity  of  the  RAZOR  EX  is  less  than  the  theoretical  limit  of  single  copy 
detection.  A  major  source  of  sensitivity  loss  is  the  initial  dilution  of  sample.  Another  specific  source  of 
sensitivity  loss  is  the  sample  splitting  into  10  individual  assay  amplifications.  Subsampling  in  this  manner 
automatically  reduces  sensitivity  by  10-fold,  and  provides  for  a  sharp  increase  in  false  negatives  because 
of  unfavorable  Poisson  distributions  at  low  concentrations  of  target  DNA.  This  can  potentially  be 
overcome  by  targeting  high-copy  number  loci,  and  would  not  be  an  issue  when  sampling  bulk  or  high- 
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titer  samples.  One  benefit  of  this  approach  is  the  reduction  in  assay  interference  normally  present  in  a 
multiplex  PCR  format. 


The  LOD  of  the  most  sensitive  assay  tested  (Y.  pestis)  is  between  50-500  genome  equivalents.  Ignoring 
plasmid  copy  number  and  allowing  for  moderate  sample  loss  due  to  bacterial  lysis  inefficiency,  the 
actual  sensitivity  in  the  individual  PCR  reactions  is  more  likely  on  the  order  of  10  genome  equivalents. 
Therefore,  the  sensitivity  of  RAZOR  EX  is  limited  by  sample  preparation  and  amplification  inefficiencies. 
Considering  the  three  sources  of  sensitivity  loss  listed  above,  the  detection  capabilities  of  the  RAZOR  EX 
are  likely  very  close  to  single  copy  limits. 
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Genedrive™ 


by  Epistem,  Inc. 


System  Specifications 


Vendor:  Epistem,  Inc. 

Website:  www.epistem.co.uk 

System  Cost:  $4,000 

Assay  Cost:  Price  Request  (Price  estimated  to  be  $85/assay, 
subject  to  quantity) 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  4  per  Assay 

Assay  Shelf  Life:  Unknown 

Sample  Size  Required:  20  pL 

Type  of  Detection:  Nucleic  Acid 

Time  to  Result:  60  minutes 

System  Weight:  1.2  lbs 

Operating  Range:  32-131  °F  (0-55  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  is  still  in  prototype  stage.  Some  components 
need  refinement. 


System  Description 


The  Genedrive  is  a  fully  integrated,  endpoint  PCR-based 
platform.  Genedrive’s  proprietary  “hybrid  thermal  engine” 
allows  faster  cooling  rates  and  shorter  annealing  times 
relative  to  conventional  PCR  devices.  The  system  is  capable 
of  performing  ultra  fast  PCR  cycling  of  30  cycles  in  as  little 
as  17  minutes  and  is  controlled  by  a  single  button.  The 
Genedrive  was  designed  to  be  a  highly  cost-effective  way 
of  moving  molecular  diagnostics  from  the  laboratory  to  the 
point  of  need  across  several  markets  including  government. 
Epistem  currently  has  a  Mycobacterium  tuberculosis 
IVD  assay  for  the  Genedrive  that  has  received  European 
approval.  For  this  testing  Epistem  developed  multiplex 
assays  to  detect  Bacillus  anthracis,  Yersinia  pestis, 

Vaccinia  and  Venezuelan  Equine  Encephalitis  and 
a  singleplex  assay  to  detect  active  Botulinum  neurotoxin 
type  A. 


Test  Bed  Review 


On  paper,  the  Genedrive’s  low  cost,  small  footprint  and 
short  run  time  were  appealing;  unfortunately,  the  assays 
did  not  meet  expectations  and  required  modified  testing 
procedures  to  achieve  successful  data.  After  several 
iterations,  including  eliminating  the  sample  preparation 
paper  and  making  internal  hardware  structure  changes,  the 
system  was  able  to  detect  approximately  1.66xl05 

CFU/mL  in  the  Yersinia  pestis  assay.  However,  Bacillus 
anthracis  and  Vaccinia  detection  limits  were  100  fold 
higher  and  the  device  couldn’t  run  the  VEE  assay  because 
of  firmware  shortcomings.  The  60  minute  run  time  was 
unexpected,  since  the  system  was  projected  to  perform 
30  cycles  in  17  minutes.  Genedrive  also  produced 
false  positives  and  inconsistent  results.  Our  testing 
demonstrated  that  unrefined  sample  preparation  and  assay 
reagents  also  contributed  to  poor  performance. 

Field-users  were  impressed  with  the  size  of  the  system, 
but  little  else.  Surprisingly,  the  sample  preparation  created 
the  hardest  task  of  manipulating  several  tubes.  One  user 
commented,  “Anything  that  requires  several  steps  would 
require  two  operators  for  set  up  and  therefore  would  not 
be  used  in  the  field”.  One  soldier  noted  that  the  humid 
weather  prevented  the  sample  preparation  cards  from 
completely  drying.  Another  user  commented  “The  device 
needs  some  method  of  data  accessibility  and  storage  such 
as  a  thumb  drive  for  chain  of  evidence.” 

Genedrive  did  not  perform  well  for  bacterial  and  virus 
detection  but  was  the  only  PCR-based  system  that 
could  detect  the  presence  of  toxin.  The  system  design  is 
promising  and  grabs  one’s  attention,  but  assay  design  and 
sample  preparation  require  additional  development,  putting 
this  system  near  the  bottom  for  performance  in  our  testing. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  GE,  Virus  =  GE,  toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

10 

O 

69,000 

Yersinia  pestis 

10 

O 

200 

Vaccinia 

10 

O 

34,000 

Venezuelan  Equine 
Encephalitis 

10 

O 

Not  Tested 

Clostridium 

Botulinum  Toxin 

No  Claim 

O 

10,000 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

69,000 

O 

N/A 

Yersinia  pestis 

200 

O 

N/A 

Vaccinia 

34,000 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

10,000 

O 

N/A 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 

The  Genedrive™  is  a  fully  integrated,  endpoint  PCR-based  platform. 

Genedrive's  proprietary  "hybrid  thermal  engine"  allows  faster  cooling 
rates  and  shorter  annealing  times  relative  to  conventional  PCR 
devices.  The  system  is  capable  of  performing  ultra-fast  PCR  cycling  of 
30  cycles  in  as  little  as  17  minutes  and  is  controlled  by  a  single  button. 

The  Genedrive  was  designed  to  be  a  highly  cost-effective  way  of 
moving  molecular  diagnostics  from  the  laboratory  to  the  point  of 
need  across  several  markets  including  government.  Epistem  has  a 
Mycobacterium  tuberculosis  in  vitro  diagnostics  (IVD)  assay  for  the 
Genedrive  that  received  the  CE-IVD  Mark  in  Europe.  For  this  testing, 

Epistem  developed  multiplex  assays  to  detect  Bacillus  anthracis , 

Yersinia  pestis,  Vaccinia  and  Venezuelan  Equine  Encephalitis  and  a  singleplex  assay  to  detect  active 
Botulinum  toxin  type  A. 

For  detection  of  agent  with  Epistem's  BIOT  ID™  Cartridges,  25pL  of  sample  is  spotted  onto  a  1cm  disc  of 
2B  BlackBio  BlackLight  paper  and  allowed  to  dry  for  10  minutes.  Three  1mm  discs  are  removed  from  the 
spot  with  a  biopsy  punch  and  one  disc  is  added  to  each  well  of  the  cassette.  Additionally,  three  GE 
Healthcare  Ready-To-Go  PuReTaq  PCR  beads  are  rehydrated  with  20pL  water  and  each  transferred  to  a 
well  of  the  cassette.  The  cassettes  contain  lyophilized  primer/probe  oligonucleotides  in  two  of  the  three 
wells,  while  the  third  well  is  a  negative  control.  Each  of  the  wells  containing  oligonucleotides  contained 
sets  allowing  detection  of  two  agents.  Preparation  of  the  sample  using  the  BlackLight  paper,  rehydration 
of  PCR  beads,  and  assembly  of  the  cassettes  takes  approximately  15  minutes.  Once  the  cassette  is 
loaded  into  the  device,  results  are  ready  in  65  minutes.  The  device  displays  "undetected  OK"  or 
"detected  positive."  The  user  must  then  navigate  to  a  secondary  screen  and  determine  whether  the 
melt  peaks  identified  by  the  software  match  the  expected  Tm  (melting  temperature)  although  the 
system,  as  configured,  does  not  prompt  the  user  to  view  the  data. 

Early  in  the  evaluation,  potential  problems  with  external  light  interference,  sample  preparation  and 
assay  stability  were  identified.  To  overcome  the  light  issue,  the  vendor  replaced  the  Genedrive  with 
another  model  that  contained  an  additional  gasket  to  seal  the  internal  components.  The  vendor  also 
identified  the  lyophilization  of  the  BIOT  ID  reagents  as  being  potentially  defective.  To  remedy  the  assay 
stability  problems,  the  vendor  provided  individual  primer  and  probe  mixtures  for  each  assay  to  replace 
the  BIOT  ID  cartridges.  Further  testing  demonstrated  the  BlackLight  paper  was  an  inefficient  sample 
preparation  medium  and  caused  significant  dilution  of  the  target. 

The  testing  procedures  were  modified  as  a  result  of  these  initial  tests.  Sample  preparation  using  the 
BlackLight  paper  was  replaced  by  testing  of  purified  DNA  extracted  from  inactivated  agents  and  assays 
were  run  in  cassettes  not  containing  lyophilized  primer/probes.  Agent  specific  primers  and  probes  were 
added  from  a  rehydrated  stock.  DNA  had  been  purified  from  each  agent  in  order  to  test  the  Tetracore  T- 
COR  4  system,  so  these  DNA  samples  were  used  to  test  the  sensitivity  of  the  Genedrive  assays.  For  each 
test,  three  Ready-To-Go  PuReTaq  PCR  beads  were  rehydrated  with  20pL  water  each  and  transferred  to  a 
well  of  the  cassette.  One  microliter  of  DNA  in  TE  buffer  and  one  microliter  of  primer/probe  mix  in  water 
were  added  to  each  cassette  well.  Assays  were  considered  positive  only  if  the  melt  peak  matched  the 
expected  Tm  provided  by  Epistem. 
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The  BoNT  A  assay,  as  stated  above,  is  a  fluorescence-based  assay  based  on  the  enzymatic  activity  of 
BoNT  A.  The  amount  of  fluorescence  is  directly  proportional  to  the  amount  of  active  BoNT  A  in  the 
sample.  The  assay  developed  by  SRC  (Syracuse  Research  Corporation)  is  a  simple,  straightforward  assay 
in  which  the  user  mixes  the  sample  into  three  assay  tubes  containing  a  synthetic,  fluorescently  labeled 
substrate.  The  results  are  available  after  a  one  hour  isothermal  incubation  at  37°C  as  an  increase  in 
fluorescence  relative  to  the  control  tube.  Unfortunately,  neither  SRC  nor  Epistem  provided  guidelines  for 
threshold  A-fluorescence  measurements. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Purified  DNA  extracted  from  gamma-irradiated  Bacillus  anthracis  spores  was  serially  diluted  in  10-fold 
increments  into  50  millimolar  (mM)  Tris  HCI,  pH  8.0  and  evaluated  on  the  Genedrive  platform.  PCR 
beads  were  rehydrated  with  20pL  water  then  lpL  primer/probe  mix  and  lpL  sample  was  added  after 
transferring  the  reaction  mix  to  cassette.  The  LOD  of  the  device  was  determined  to  be  nearly  70,000  GE, 
7,000-fold  greater  than  the  LOD  claimed  by  Epistem.  Considering  the  size  of  the  sample  and  reported  GE 
to  CFU  ratio,  this  LOD  is  equivalent  to  approximately  2.90xl07  CFU/mL. 


Table  16.  Bacillus  anthracis  LOD 


Concentration 

(picogram 

(Pg)M) 

Total  pg 

Total  Genome 
Equivalents 
(GE) 

Equivalent 

CFU/mL 

Results  by  Loci 
(Positives/Total  Runs) 

rpoB 

3.90x10  1 

3.90x10  1 

6.90X101 

2.90xl04 

0/1 

3.90x10° 

3.90x10° 

6.90xl02 

2.90x10s 

0/1 

3.90X101 

3.90X101 

6.90xl03 

2.90xl06 

1/3 

3.90xl02 

3.90xl02 

6.90xl04 

2.90xl07 

3/3 

Vendor  Claimed  LOD:  10GE 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Purified  DNA  extracted  from  gamma-irradiated  Yersinia  pestis  cells  was  serially  diluted  in  10-fold 
increments  into  50mM  Tris  HCI,  pH  8.0  and  evaluated  on  the  Genedrive.  The  LOD  of  the  device  was 
determined  to  be  over  200  copies,  20-fold  greater  than  the  LOD  claimed  by  Epistem.  Considering  the 
size  of  the  sample  and  reported  GE  to  CFU  ratio,  this  LOD  is  equivalent  to  approximately  1.66xl05 
CFU/mL. 


Table  17.  Yersinia  pestis  LOD 


Concentration 

(pg/uL) 

Total  pg 

Total  GE 

Equivalent 

CFU/mL 

Results  by  Loci 
(Positives/Total  Runs) 

pPCP 

1.20X101 

1.20X101 

2.30X101 

1.66xl04 

1/3 

1.20x10° 

1.20x10° 

2.30xl0z 

1.66xl05 

3/3 

1.20X101 

1.20X101 

2.30xl03 

1. 66x10° 

3/3 

1.20xl02 

1.20xl02 

2.30xl04 

1.66xl07 

1/1 

Vendor  Claimed  LOD:  10  GE 
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Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Purified  DNA  extracted  from  gamma-irradiated  VAC  viral  particles  was  serially  diluted  in  10-fold 
increments  into  50mM  Tris  HCI,  pH  8.0  and  evaluated  on  the  Genedrive.  Two  rounds  of  testing  were 
completed  because  Epistem  miscommunicated  the  expected  Tm  .  Since  the  device  has  three  wells,  the 
lowest  two  concentrations  were  evaluated  simultaneously.  The  gain  for  well  three  appears  to  the  set 
much  higher  than  wells  one  and  two;  therefore,  the  6.90xl0~2pg/pL  sample  (run  in  well  three)  was 
positive  while  the  6.90x10  1  pg/pL  sample  (run  in  well  two)  was  negative.  The  LOD  of  the  device  was 
determined  to  be  6.90x10°  pg/pL  because  this  concentration  could  be  detected  in  both  wells  two  and 
three.  Well  one  was  reserved  for  an  NTC  in  all  experiments.  This  LOD  is  equivalent  to  approximately 
34,000  copies,  over  3,000-fold  greater  than  the  LOD  claimed  by  Epistem.  Considering  the  size  of  the 
sample  and  reported  GE  to  PFU  ratio,  this  LOD  is  equivalent  to  approximately  2.35xl07  PFU/mL. 


Table  18.  Vaccinia  LOD 


Concentration 

(pg/uL) 

Total  pg 

Total  GE 

Equivalent 

PFU/mL 

Results  by  Loci 
(Positives/Total  Runs) 

pPCP 

6.90x10 2 

6.90x10 2 

3.38xl02 

2.35x10s 

i/i 

6.90x10  1 

6.90x10  1 

3.38xl03 

2.35xl06 

2/3 

6.90x10° 

6.90x10° 

3.38xl04 

2.35xl07 

3/3 

6.90X101 

6.90X101 

3.38xl05 

2.35x10s 

2/2 

6.90xl02 

6.90xl02 

3.38xl06 

2.35xl09 

3/3 

6.90xl03 

6.90xl03 

3.38xl07 

2.35xl010 

2/2 

7.50xl04 

7.50xl04 

3.38x10s 

2.35X1011 

3/3 

Vendor  Claimed  LOD:  10  GE 


Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

The  VEE  assay  was  not  run  due  to  lack  of  a  pre-programmed  RT-PCR  step  in  the  thermal  cycling 
algorithm. 
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Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

BoNT  A  complex  was  serially  diluted  into  PBS  then  tested  with  the  SRC-developed  toxin  activity  assay. 
Twenty  microliters  of  each  sample  was  added  to  each  assay  tube  (denoted  C,  Tl,  and  T2),  each 
containing  50pL  of  reagent.  The  reagent  in  the  C  tube  was  a  non-fluorescent  control  while  tubes  Tl  and 
T2  contained  a  synthetic  substrate  that  fluoresces  when  proteolytically  cleaved  by  BoNT  A.  Twenty 
microliters  of  each  sample  mixture  was  transferred  to  a  Genedrive  cassette  and  evaluated  for  increase 
in  fluorescence  during  a  1  hour  isothermal  incubation  at  37°C.  The  LOD  of  the  assay  for  BoNT  A  was 
determined  to  be  lOpg/mL  by  evaluating  the  slope  of  the  fluorescence  plots. 


Table  19.  Clostridium  botulinum  toxin  Limit  of  Detection  LOD 


Concentration 

(nanogram 

(ng)/mL) 

Total  ng 

Call 

Slope  Values  (AF/At) 

Control 

Tl 

T2 

l.OOxlO2 

l.OOxlO1 

0/1 

-1.06 

-0.14 

-0.05 

l.OOxlO3 

1.00x10° 

0/1 

-1.00 

-0.04 

0.02 

l.OOxlO4 

l.OOxlO1 

3/3 

-.092T 

1.21T 

0.93T 

T  Average  of  Triplicates 
Vendor  Claimed  LOD:  N/A 


Multiplex  Detection 

Manufacturing  issues  with  the  BIOT  ID  cartridges  were  identified  early  in  the  device  assessment; 
therefore,  all  testing  was  completed  with  rehydrated  primer/probe  mixtures  in  a  singleplex  format.  As 
tested,  no  multiplex  data  was  generated. 

Discussion 

Call  Assignments 

Individual  call  assignments  are  made  based  upon  whether  products  can  be  detected  by  automated, 
high-resolution  melt  curve  analysis  (peak  negative  derivative  of  the  fluorescence  versus  temperature 
curve).  If  products  are  detected  by  the  algorithm,  the  device  displays  "detected  positive"  on  the  screen. 
However,  the  user  is  required  to  navigate  to  a  secondary  screen  which  displays  the  melting 
temperatures  of  the  products  and  compare  the  temperatures  to  parameters  supplied  by  the  vendor. 
This  creates  issues  wherein  spurious,  non-specific  amplifications  are  detected  by  the  algorithm.  This 
triggers  the  device  to  display  a  positive  detection  on  the  results  screen  but  the  Tm  of  the  product  does 
not  match  the  expected  temperature  range-thus  resulting  in  a  false  positive  result.  For  the  purposes  of 
this  evaluation,  only  positive  instrument  results  with  a  correct  amplicon  Tm  were  considered  positive  for 
the  test  agent. 

Plasmid  and  Chromosomal  PCR  Targets 

For  the  two  bacterial  organisms,  the  Epistem  Genedrive  assays  use  both  chromosomal  (B.  onthrocis)  and 
plasmid  (Y.  pestis)  targets.  Each  approach  has  benefits  and  weaknesses;  plasmids  tend  to  be  present  at 
higher  copy  number  and  therefore  have  better  sensitivity,  while  well-selected  chromosomal  markers 
tend  to  be  less  prone  to  cross  reactivity  due  to  horizontal  gene  transfer.  Using  both  has  distinct 
advantages,  giving  high  confidence  detection  and  identifying  genetic  elements  required  for  virulence  in 
a  single  instrument  run.  For  the  viral  targets  (VEE  E2  and  VAC  F5L),  these  are  both  single-copy  targets  by 
definition,  and  specificity  would  improve  with  multiple  independently  validated  assays. 
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The  choice  of  the  rpoB  gene  for  B.  anthracis  detection  can  be  problematic  in  terms  of  specificity,  as  it  is 
essentially  a  housekeeping  gene  found  in  most,  if  not  all,  bacteria.  In  fact,  like  16S  sequences,  small 
variations  in  rpoB  sequence  are  often  used  for  taxonomic  discrimination  of  bacteria.  However,  this  was  a 
conscious  decision  by  Epistem  so  that  near  neighbors  could  be  identified  in  environmental  samples  to 
distinguish  from  false  positives.  The  expected  melting  temperatures  of  amplicons  from  the  near 
neighbors  were  provided. 

The  choice  of  a  pPCPl  plasmid  target  for  Y.  pestis  detection  is  expected  to  be  more  sensitive  than  a 
chromosomal  target,  given  the  known  higher  target  copy  number  present.  This  is  reflected  in  the  data 
presented,  which  is  more  than  ten-fold  more  sensitive  than  the  B.  anthracis  chromosomal  assay.  The 
tradeoff  is  specificity,  as  plasmids  (including  pPCPl)  are  often  shared  among  different  species  in  the 
same  genus  as  well  as  other  bacteria  in  a  common  community. 

It  would  appear  that  there  is  substantial  room  for  improvement  in  the  bacterial  assays,  given  the 
sensitivity  is,  at  best,  100  GE  per  reaction.  In  this  case,  it  reflects  103  or  more  target  copies  for  pPCPl  per 
reaction  and  is  significantly  less  sensitive  than  the  theoretical  single  copy  LOD.  It  is  not  known  whether 
the  general  lack  of  sensitivity  compared  with  other  systems  is  due  to  poor  assay  performance  or  is  a 
limitation  of  the  amplification  and  detection  system.  With  a  wider  set  of  better-designed  assays,  the 
flexibility  of  the  system  could  be  utilized  in  either  "screening"  (many  singleplex  assays  for  multiple 
organismal  targets)  or  "confirmation"  (multilocus  testing  for  a  single  organism)  modes.  This  would  allow 
for  rapid  threat  discrimination  with  high-confidence  results  in  the  field. 

Assay  Sensitivity 

Assay  sensitivity  was  as  high  as  100-fold  less  than  the  company's  claimed  10  copies  cloned  into  a 
plasmid  vector.  This  lack  of  sensitivity  is  compounded  by  the  relatively  small  template  input  volumes, 
driving  up  the  concentration  of  the  sensitivity  LOD  up  to  105-107  copies  per  mL.  Using  template  to 
rehydrate  the  lyophilized  products  could  increase  the  sensitivity  of  concentration  by  20-fold,  but  would 
not  be  expected  to  increase  the  overall  sensitivity  measured  by  genomics  equivalents  per  reaction. 

The  novel  Botulinum  toxin  assay  designed  for  Epistem  by  SRC  produced  sensitivity  equivalent  to  that  of 
a  standard  sandwich  immunoassay.  The  assay  functions  by  the  specific  cleavage  of  a  proprietary 
synthetic  fluorophore-tagged  peptide  that  when  cleaved  results  in  fluorescence  transfer.  The  Genedrive 
was  the  only  PCR  platform  able  to  provide  direct  toxin  detection.  Additional  software  development  is 
needed  in  order  to  provide  objective  automated  assay  calls,  which  for  this  study  were  made  by  the 
operator  using  visual  inspection  of  fluorescent  curves. 

The  Genedrive  system  represents  a  portable  and  point  of  care  molecular  detection  system.  In  addition, 
three  separate  reaction  chambers  give  the  user  the  option  of  either  running  three  different  reactions  in 
parallel  to  provide  more  agent  coverage,  or  running  identical  reactions  in  triplicate  to  provide  higher 
confidence.  Running  assays  in  this  manner  precludes  the  opportunity  of  including  controls  for  the  assay. 

At  the  time  of  this  evaluation,  the  system  could  not  be  considered  COTS  technology,  but  rather  a  late 
beta  test.  There  were  significant  issues  with  routine  functions,  such  as  pre-programmed  RT-PCR  assays 
and  the  lack  of  integrated  battery  power.  The  provided  punch  card  sample  preparation  system  was 
virtually  non-functional  and  not  able  to  provide  an  amplification-ready  template.  This  significantly 
reduces  the  advantages  of  the  portability  of  the  system,  which  still  requires  additional  infrastructure  for 
template  preparation. 
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Liat 


TM 


by  IQuum 


System  Description 


The  Liat  is  an  automated  sample-to-result  detection 
analyzer.  IQuum’s  lab-in-a-tube  (Liat)  was  designed  to 
enable  non-specialized  personnel  to  perform  “moderate 
complexity”  tests  in  hospital  labs  or  other  near-patient 
setting.  The  Liat  has  automated  sample  processing  in  a 
flexible  tube  containing  pre-packaged  reagents.  Peristaltic 
manipulations  by  actuators  in  the  analyzer  move  the 
sample  through  each  stage  of  sample  processing  ending 
with  a  PCR  or  RT-PCR  amplification  the  sample  for 
target  identification.  Currently  IQuum  has  a  FDA  510(K) 
cleared  Liat  Influenza  A/B  assay  on  the  market  which  is 
intended  for  use  in  laboratories  certified  under  the  Clinical 
Laboratory  Improvement  Amendments  (CLIA).  For  this 
testing  IQuum  developed  two  duplex  assays,  one  that 
detects  Bacillus  anthracis  and  Yersinia  pestis  and  another 
to  detect  Vaccinia  and  Venezuelan  Equine  Encephalitis. 


System  Specifications 


Vendor:  IQuum 
Website:  www.iquum.com 
System  Cost:  $25,000.00 

Assay  Cost:  Price  on  Request  (Estimated  to  be  $60  per  test) 

Assay  Storage  Requirements:  Refrigeration 
Agents  Tested  per  Assay:  2 
Assay  Shelf  Life:  1  year  at  4  °C 
Sample  Size  Required:  200  pL 

Type  of  Detection:  Real-time  nucleic  acid  amplification  and 
fluorescence  detection 

Time  to  Result:  30  minutes 

System  Weight:  8.3  lbs 

Operating  Range:  40  - 122  °F  (4  -  50  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field. 

U  ?  .a.  a  ^ 


Test  Bed  Review 


A  combination  of  small  footprint,  automated  sample  prep, 
20  minute  run  and  sensitivity  down  to  the  single  digits 
(CFU/mL),  the  Liat  analyzer  was  a  top  performer  both  in 
the  lab  and  in  the  field.  Achieving  detection  levels  down  to 
6.5  CFU/mL  in  their  Yersinia  pestis  duplex  assay,  the  Liat’s 
sensitivity  was  superior  to  most  other  PCR  platforms  in  the 
test  bed.  In  addition  to  performing  well  in  the  presence 
of  one  agent,  IQuum  also  scored  well  in  the  multiplex 
category,  showing  little  to  no  loss  of  signal  when  samples 
were  combined  in  a  duplex  format.  During  testing,  the 
Liat  did  present  with  several  error  messages  that  required 
IQuum’s  intervention.  However,  these  errors  were  a  result 
of  system  checks  put  in  place  as  a  requirement  for  a  FDA 
approved  system. 

In  the  field,  the  end-users  were  impressed  with  the  ease 
of  set-up  and  minimum  amount  of  training  required.  One 
operator  stated  that  “This  device  is  the  easiest  piece 
of  equipment  I’ve  ever  used.”  Admittedly  the  current 
configuration  of  this  system  was  not  intended  for  outdoor 
use  and  end-users  would  like  to  see  some  modifications  to 
its  current  design.  The  size  of  the  buttons  and  the  use  of 
the  stylus  were  top  on  their  list.  One  user  commented  “The 
Login/Pin  requirement  is  overkill  for  military  applications 
and  the  stylus  would  get  lost”. 

As  a  result  of  testing  in  an  analytical  and  mobile  laboratory 
and  the  field,  the  Liat  showed  a  great  deal  of  potential. 

We  believe  with  little  investment  in  its  design  (i.e., 
ruggedization  of  the  exterior,  integrated  battery)  this  system 
would  be  a  good  fit  in  most,  if  not  all,  military  testing 
applications. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


JlNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

2.00X101 

Q 

l.OOxlO3 

Yersinia  pestis 

5.00x10° 

© 

6.5x10° 

Vaccinia 

2.00xl02 

© 

2.50xl03 

Venezuelan  Equine 
Encephalitis 

4.00xl04 

© 

2.10xl03 

Clostridium 

Botulinum  Toxin 

N/A 

o 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO3 

© 

l.OOxlO3 

Yersinia  pestis 

5.00x10° 

© 

l.OOxlO1 

Vaccinia 

2.50xl03 

© 

2.50xl04 

Venezuelan  Equine 
Encephalitis 

2.10xl03 

© 

2.10xl04 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


<u 

cn 

c 
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Overview 

IQuum's  Liat  is  an  automated  sample-to-result  detection 
analyzer.  It  was  designed  to  enable  non-specialized 
personnel  to  perform  "moderate  complexity"  tests  in 
hospital  laboratories  or  other  near-patient  setting.  The  Liat 
has  automated  sample  processing  in  a  flexible  tube 
containing  pre-packaged  reagents.  Peristaltic  manipulations 
by  actuators  in  the  analyzer  move  the  sample  through  each 
stage  of  sample  processing  ending  with  a  PCR  or  RT-PCR 
amplification  the  sample  for  target  identification.  Currently 
IQuum  has  an  FDA  510(k)  cleared  Liat  Influenza  A/B  assay 
on  the  market  that  is  intended  for  use  in  laboratories  certified  under  the  Clinical  Laboratory 
Improvement  Amendments  (CLIA).  For  this  testing  IQuum  developed  two  duplex  assays,  one  that 
detects  Bacillus  anthracis  and  Yersinia  pestis  and  another  to  detect  Vaccinia  and  Venezuelan  Equine 
Encephalitis. 

The  Liat  system  was  tested  to  evaluate  the  technology  and  verify  vendor  claims  of  LOD  for  four  biothreat 
agents  (two  bacterial  and  two  viral)  using  custom  designed  B.  anthracis/Y.  pestis  and  VEE/VAC  assays. 
The  Liat  utilizes  simple,  load-and-go  format  assays  in  which  the  user  adds  a  sample,  inserts  the  tube  into 
the  analyzer,  and  walks  away.  The  system  uses  actuators  and  peristaltic  motion  to  move  the  sample  to 
different  regions  of  the  assay  tube  where  the  nucleic  acid  is  purified  then  amplified.  Sample  injection 
into  the  cassette  takes  seconds.  The  run-time  of  the  B.  anthracis/Y.  pestis  assay  is  19  minutes  while  the 
run-time  of  the  VEE/VAC  assay  is  32  minutes.  The  time  discrepancy  between  the  assays  is  due  to  the 
inclusion  of  a  reverse  transcription  step  in  the  VEE/VAC  assay.  The  Liat  is  designed  to  be  used  in  an  FDA 
regulated  laboratory  environment;  therefore,  several  of  its  features  were  considered  hindrances  for 
field  or  mobile  laboratory  use.  Two  specific  examples  are  its  narrow  temperature  operating  range  and 
requirement  that  users  log-in  with  a  username  and  PIN. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Gamma-irradiated  Bacillus  anthracis  spores  were  diluted  to  lxlO1,  lxlO2,  and  lxlO3  CFU/mL  in  PBS  then 
tested,  in  triplicate,  using  the  B.  anthracis/Y.  pestis  duplex  assay.  Each  200pL  sample  was  transferred 
directly  into  the  tube  and  the  tube  was  inserted  immediately  into  the  device.  The  vendor-stated  LOD  of 
this  assay  is  2xlOJ  CFU/mL,  although  100  times  more  sample  was  required  for  consistent  detection  in 
this  study.  When  testing  began,  two  samples  were  tested  at  10  CFU/mL  and  both  were  detected  by  the 
Liat  device.  Complete  testing  was  not  performed  until  two  and  a  half  months  later.  By  that  time 
sensitivity  of  the  assays  had  diminished  by  a  factor  of  100.  During  that  time,  the  operator  also  received 
invalid  PCR  result  errors  during  the  testing.  An  "invalid"  PCR  result  is  reported  when  either  the  Crossing 
Threshold  (CT)  of  the  internal  control  is  outside  specification  or  the  shape  of  the  real-time  data  does  not 
fit  acceptance  criteria.  Issues  of  assay  stability  may  be  responsible  for  the  decrease  in  performance.  Final 
LOD  of  the  B.  anthracis  assay  was  determined  to  be  1.0xl03CFU/ml. 
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Table  20.  Bacillus  anthracis  LOD 


Concentration 
(Colony  Forming 
Units  (CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents  (GE) 

Results  by  Loci 
(Positives/Total  Runs) 

Lethal  Factor 

l.OOxlO1* 

2.00x10° 

4.76x10° 

0/1 

l.OOxlO2 

2.00X101 

4.76X101 

0/1 

l.OOxlO3** 

2.00xl02 

4.76xl02 

3/4 

*  First  tests  were  positive  for  111  samples  at  10  CFU,  may  indicate  reagent  stability  issue 
**  Data  includes  "invalid"  PCR  result 
Vendor  Claimed  LOD:  2.00xl01CFU/mL 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma-irradiated  Yersinia  pestis  cells  were  diluted  to  6.5x10°,  6.5X101,  and  1.3xl02  CFU/mL  in  PBS  then 
tested,  in  triplicate,  using  the  B.  anthracis/Y.  pestis  duplex  assay.  Each  200pL  sample  was  transferred 
directly  into  the  tube  and  the  tube  was  inserted  immediately  into  the  device.  The  vendor  stated  LOD  of 
this  assay  is  5x10°  CFU/mL,  and  this  LOD  was  confirmed  in  this  study. 


Table  21.  Yersinia  pestis  LOD 


Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

Pesticin 

6.50x10° 

1.30x10° 

1.80x10° 

3/3 

6.50X101 

1.30X101 

1.80X101 

3/3 

1.30xl02 

2.60X101 

3.60X101 

3/3 

Vendor  Claimed  LOD:  5.00x10°  CFU/mL 


Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  was  diluted  to  2.5xl02  and  2.5xl03  PFU/mL  in  PBS  then  tested,  in  triplicate,  using 
the  VAC/VEE  duplex  assay.  Each  200pL  sample  was  transferred  directly  into  the  tube  and  the  tube  was 
inserted  immediately  into  the  device.  The  vendor  stated  LOD  of  this  assay  is  250  PFU/mL,  although  10 
times  more  sample  was  required  for  consistent  detection  in  this  study.  One  of  the  2.50xl02  PFU/mL 
samples  was  reported  invalid.  An  "invalid"  result  is  reported  when  either  the  CT  of  the  internal  control  is 
outside  specification  or  the  shape  of  the  real-time  data  does  not  fit  acceptance  criteria.  The  LOD  of  this 
assay  was  determined  to  be  2.50xl03  PFU/mL. 


Table  22.Vaccinia  LOD 


Concentration 
(Plaque  Forming 
Units  (PFU)/mL) 

Total  PFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

Pesticin 

2.50xl02* 

5.00X101 

7.21X101 

1/3 

2.50xl03 

5.00xl02 

7.21xl02 

3/3 

*lncludes  an  "invalid"  PCR  result 
Vendor  Claimed  LOD:  2.00xl02  PFU/mL 
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Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

Gamma-irradiated  VEE  viral  particles  were  diluted  to  2.1xl02,  2.1xl03,  and  2.1xl04  PFU/mL  in  PBS  then 
tested,  in  triplicate,  using  the  VAC/VEE  duplex  assay.  Each  200pL  sample  was  transferred  directly  into 
the  tube  and  the  tube  was  inserted  immediately  into  the  device.  The  vendor  stated  LOD  of  this  assay  is 
4.00xl04  PFU/mL.  This  evaluation  found  that  the  LOD  was  20-fold  better  than  reported  by  the 
manufacturer.  The  LOD  of  this  assay  was  determined  to  be  2.10xl03  PFU/mL. 


Table  23.  Venezuelan  Equine  Encephalitis  LOD 


Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results  by  Loci 
(Positives/Total  Runs) 

Pesticin 

2.10xl02 

4. 20x10s 

4.20x10s 

0/3 

2.10xl03 

4.20xl02 

4.20xl02 

3/3 

2.10xl04 

4.20x10s 

4.20x10s 

3/3 

*VEE  provider  did  not  quantitate  a  GE  to  PFU  ratio.  A  ratio  of  one  was  used  for  this  study. 
Vendor  Claimed  LOD:  4.00xl04  PFU/mL 


Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

No  assay  was  developed  for  C.  botulinum  toxin  or  DNA. 

Multiplex  Detection 

Gamma-irradiated  B.  onthrocis  spores  and  Y.  pestis  cells  were  each  diluted  into  PBS  then  mixed  to  create 
samples  with  1,000  CFU/mL  B.  anthrocis  and  6.5  CFU/mL  Y.  pestis.  The  first  mixed  sample  was  tested 
and  both  agents  were  detected.  In  the  second  run,  Y.  pestis  was  not  detected.  In  accordance  with 
established  protocols,  fresh  samples  were  created  with  1,000  CFU/mL  B.  onthrocis  and  65  CFU/mL  Y. 
pestis.  This  sample,  with  10  times  more  Y.  pestis  was  tested  in  triplicate.  Both  agents  were  detected  in 
all  three  replicates. 

Gamma-irradiated  VAC  and  VEE  viral  particles  were  each  diluted  into  PBS  then  mixed  to  create  samples 
with  2,500  PFU/mL  VAC  and  2,100  PFU/mL  VEE  and  tested  in  triplicate  with  the  VAC/VEE  duplex  assay. 
VEE  was  not  detected  in  any  of  the  three  assays;  therefore  fresh  samples  containing  10-fold  more  VEE 
were  created  and  tested  in  triplicate.  In  these  replicates,  VEE  was  detected  in  all  three  tests  while  VAC 
was  only  detected  in  two  of  the  three  assays.  Again,  the  concentration  of  the  agent  not  yielding 
consistent  detection,  in  this  case  VAC,  was  increased  10-fold.  The  samples,  now  containing  25,000 
PFU/mL  VAC  and  21,000  PFU/mL  VEE,  were  tested  in  triplicate.  Both  targets  were  detected  in  all  three 
assays. 
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Table  24.  Multiplex  Evaluation 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

B.  anthracis 

l.OOxlO3 

2.00xl02 

4.76xl02 

5/5 

Y.  pestis 

6.50x10° 

1.30x10° 

1.77x10° 

1/2 

6.50X101 

1.30X101 

1.77X101 

3/3 

Agent 

Concentration 

(PFU/mL) 

Total  PFU 

Total  EG 

Results 

(Positives/Total  Runs) 

VAC 

2.50x10s 

5.00xl02 

7.20xl02 

5/6 

2.50xl04 

5.00xl03 

7.20xl03 

3/3 

VEE 

2.10xl03 

4.20xl02 

4.20xl02 

2/3 

2.10xl04 

4.20xl03 

4.20x10s 

6/6 

*Provider  of  VEE  did  not  quantitate  a  GE  to  PFU  ratio.  For  the  purpose  of  this  study  a  ratio  of  1  was  used. 


Discussion 
Call  Assignments 

All  individual  system  calls  are  made  for  each  individual  organism  from  a  single  locus  PCR  amplification. 
The  tested  format  was  duplex  assays  pairing  B.  anthracis  and  Y.  pestis  or  VEE  and  VAC,  each  of  which 
includes  an  internal  positive  amplification  control. 

Each  PCR  amplification  curve  is  processed  with  an  algorithm  to  determine  CT  values  and  whether  end¬ 
point  PCR  fluorescent  values  are  within  an  acceptable  range,  which  determines  whether  a  given  call  is 
"positive"  or  "negative."  Additional  curve  fitness  parameters  are  also  evaluated  to  insure  that  the 
amplification  fits  an  acceptable  logarithmic  profile.  Any  positive  result  based  upon  fluorescence 
measurements  but  failing  the  curve  profile  metrics  is  called  as  "indeterminate."  If  all  three  PCR  profiles 
(two  target  organisms  and  internal  control)  fail  the  curve  fitness  determination,  the  entire  assay  is  called 
as  "invalid."  Finally,  the  internal  control  must  be  determined  to  be  positive  in  order  to  make  a 
"negative"  call  for  both  target  organisms:  otherwise  an  "invalid"  assay  call  is  made.  However,  a  positive 
result  with  either  of  the  target  organisms  may  inhibit  the  amplification  of  the  internal  control  target, 
presumably  due  to  consumption  of  deoxyribonucleotide  triphosphates  and  monopolization  of 
polymerase.  Therefore,  any  positive  result  from  one  of  the  three  PCR  reactions  with  appropriate  curve 
metrics  is  considered  a  positive  amplification  control.  While  this  is  certainly  an  acceptable  theoretical 
assumption,  perhaps  a  better  theoretical  solution  (particularly  for  any  assay  undergoing  rigorous  review, 
such  as  an  FDA  submission  for  clinical  applications)  would  be  to  develop  a  more  rigorous  internal 
amplification  control. 

There  were  multiple  ambiguous  PCR  calls  made  that  did  not  specifically  correlate  with  template 
concentrations  near  the  LOD.  The  nature  of  these  calls  is  not  known,  but  potential  sources  of  this  error 
may  include: 

•  Systemic  errors  during  software  analysis  of  raw  data 

•  Reagent  instability  or  variability 

•  Intermittent  hardware  fault  of  the  specific  instrument  used  for  this  evaluation 

•  Hardware  design  flaw  inherent  to  the  manufactured  product 
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These  issues  will  need  to  be  addressed  in  any  acquisition  program  in  order  to  ensure  robust  and 
consistent  performance,  especially  in  challenging  environments  outside  of  a  controlled  laboratory 
setting. 

Plasmid  and  Chromosomal  Targets 

The  single  locus  detections  scheme  employed  in  the  two  assays  does  not  allow  for  multiplex  detection  of 
both  chromosomal  and  multiple  plasmid  PCR  targets  routinely  used  for  high  confidence  and 
simultaneous  detection  of  the  threat  agents  and  virulence  islands  required  for  pathogenesis. 
Presumably,  the  more  highly  multiplex  formats  claimed  but  not  demonstrated  (16-plex  based  upon 
independent  and  spectrally  separated  excitation  and  emission  channels)  and  parallel  processing  could 
produce  the  required  data  density  for  this  level  of  sophisticated  and  conclusive  multi-agent  detection. 
However,  this  capability  was  not  tested  in  this  laboratory  evaluation. 

For  both  of  the  bacterial  targets,  the  PCR  loci  were  located  on  plasmids  (Lethal  Factor  on  pXOl  for  B. 
anthracis  and  Pesticin  on  pPCP-1  for  Y.  pestis).  While  plasmid  targets  tend  to  be  present  in  higher 
concentrations  than  chromosomal  markers,  making  them  attractive  for  sensitive  detection,  both  have 
been  known  to  be  to  be  present  in  other  members  on  the  same  genus.  Intact  or  partial  pXOl  is  often 
seen  in  other  pathogenic  group  I  Bacilli  such  as  B.  cereus.  In  the  case  of  Pesticin,  it  is  not  only  found  in 
other  Yersinia  but  closely  related  homologs  are  seen  widely  throughout  the  gram-negative  enterics.  All 
of  these  potentially  cross-reacting  organisms  also  cause  human  disease  and  are  common  in 
environmental  samples.  For  the  purposes  of  the  technical  evaluation  of  equipment  in  this  study,  this  is 
not  a  pertinent  issue. 

Assay  Sensitivity 

Assay  sensitivity  has  been  demonstrated  to  be  as  low  as  <10  CFU  with  100%  detection.  The  excellent 
sensitivity  is  in  part  due  to  the  robust  sample  preparation,  which  includes  a  relatively  large  input  sample 
volume.  This  entire  nucleic  acid  preparation  is  included  into  a  single  multiplexed  PCR  reaction  rather 
than  having  to  parse  the  purified  sample  into  multiple  parallel  reactions.  This  is  a  significant  advantage 
in  terms  of  sensitivity,  especially  when  surveying  for  multiple  target  organisms. 

However,  there  were  a  couple  of  observed  limitations  to  the  existing  configuration  that  indicated  that 
optimal  sensitivity  was  not  achieved.  First,  sensitivity  between  instrument  runs  varied  as  widely  as  100- 
fold,  which  was  attributed  to  reagent  instability  and/or  inconsistency.  Second,  only  very  low  levels  of 
agent  and  loci  multiplexing  were  tested.  Even  at  this  level,  there  was  evidence  of  unbalanced  competing 
PCR  reactions,  10-100  fold  variability  in  sensitivity  of  DNA  targets,  and  10-fold  loss  of  sensitivity  in 
multiplex  format.  Finally,  deeper  multiplex  assays  were  not  evaluated  and  remain  an  unknown  capability 
in  terms  of  sensitivity. 
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T-COR4™ 


by  Tetracore,  Inc. 


System  Specifications 


Vendor:  Tetracore,  Inc. 

Website:  www.tetracore.com 

System  Cost:  $38,500.00 
Assay  Cost:  $16,000.00 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  One.  Multiple  formats  available 

Assay  Shelf  Life:  1  year  at  room  temperature 

Sample  Size  Required:  3  pL 

Type  of  Detection:  Nucleic  Acid 

Time  to  Result:  45  minutes 

System  Weight:  6.2  lbs  (w/o  required  centrifuge) 

Operating  Range:  39-113  °F  (4-50  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field. 


System  Description 


The  T-COR  4  is  a  portable  Real-Time  PCR  thermocycler 
with  four  independent  sample  wells  capable  of  2  color 
detection.  The  T-COR  4  is  a  field  deployable  battery 
powered  system,  but  lacks  sample  preparation  capabilities. 
Tetracore’s  real-time  PCR  reagents  are  stored  at  room 
temperature  with  20+  assays  currently  available.  The 
system  in  encased  in  a  heavy  duty  protective  rubber  sleeve 
with  an  internal  8  hour  battery  for  the  thermal  cycler.  For 
this  testing,  Tetracore’s  Bacillus  anthracis  pXOl  assay 
and  Yersinia  pestis  assay  were  evaluated.  Each  assay 
contains  reagents  for  specific  target  detection  using  the 
FAM  fluorophore  and  an  internal  control  detected  with  CY5 
fluorophore. 


Test  Bed  Review 


The  T-COR  4  is  small,  lightweight  and  battery  operated, 
making  it  appealing  for  the  field.  The  main  shortcomings 
were  the  number  of  targets  (1)  per  test  and  lack  of 
integrated  sample  preparation.  With  purified  samples,  the 
system  scores  high  in  the  laboratory,  sensitivities  in  the 
femtogram  range.  The  20  minute  runtime  and  real  time 
viewing  are  appealing,  unfortunately  without  on  board 
sample  preparation,  the  scores  are  lower  than  the  other 
systems.  Additionally,  Tetracore  was  only  able  to  provide 
2  of  the  5  requested  targets,  with  no  ability  to  design 
new  assays  for  this  testing.  The  test  results  could  not  be 
saved  on  the  device,  as  configured.  Tetracore  can  provide 
software  to  operate  the  instrument,  and  save  and  analyze 
data  using  an  external  computer. 

In  the  field,  the  end-users  liked  the  ease-of-use  of 
the  system  and  had  no  trouble  running  this  system 
independently.  Although  one  soldier  commented,  “It 
doesn’t  really  add  anything  additional”,  another  added, 

“It’s  quick  to  start  up  and  easy  to  use”.  End  users  liked 
that  the  results  were  easy  to  view  and  interpret.  Consistent 
results  were  an  additional  plus  of  this  system.  Training  time 
was  minimal  and  the  consumables  were  easy  to  handle  in 
MOPP  gear.  The  small  centrifuge  for  the  PCR  tubes  added 
to  the  footprint  for  field  operators. 

As  a  result  of  testing  in  an  analytical  and  mobile  laboratory 
and  the  field  the  T-COR  4  was  a  consistent  and  easy  to 
use  system.  The  lack  of  onboard  sample  preparation  is  a 
negative,  but  the  system  is  a  reliable  workhorse. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  fg/uL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

No  Claim 

O 

3.2X101 

Yersinia  pestis 

No  Claim 

O 

1.3X101 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

3.2x10 1 

O 

N/A 

Yersinia  pestis 

1.3X101 

O 

N/A 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 

rated  by  Army  Subject  Matter  Experts. 

„AA  100 
100 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 


Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 

The  T-COR  4  is  a  portable  Real-Time  PCR  thermocycler  with  four 
independent  sample  wells  capable  of  two-color  detection.  The 
T-COR  4  is  a  field  deployable  battery  powered  system,  but  lacks 
sample  preparation  capabilities.  Tetracore's  real-time  PCR 
reagents  are  stored  at  room  temperature  with  more  than  20 
assays  currently  available.  The  system  in  encased  in  a  heavy 
duty  protective  rubber  sleeve  with  an  internal  eight  hour 
battery  for  the  thermal  cycler.  For  this  testing,  Tetracore's  6. 

anthracis  pXOl  assay  and  Y.  pestis  assay  were  evaluated.  Each  assay  contains  reagents  for  specific  target 
detection  using  the  FAM  fluorophore  and  an  internal  control  detected  with  CY5  fluorophore. 


The  Tetracore°  T-COR  4  platform  running  the  B.  anthracis  pOX-1  or  Y.  pestis  assay  was  tested  to  evaluate 
the  technology  and  verify  vendor  claims  of  LOD  for  two  bacterial  biothreat  agents  (B.  anthracis  and  Y. 
pestis).  The  Tetracore  T-COR  4  Clostridium  botulinum  A/B  assay  was  also  utilized  in  this  evaluation  to 
determine  whether  the  system  could  detect  residual  BoNT  A  DNA  in  a  preparation  of  holotoxin. 


The  T-COR  4  contains  a  photomultiplier  and  filters  to  monitor  two  emission  wavelengths  simultaneously. 
The  presence  of  target  is  monitored  on  the  FAM  channel  while  the  Cy5  channel  is  reserved  for  the 
amplification  of  an  internal  control  preloaded  and  lyophilized  in  the  assay  tubes.  Thermal  cycling  was 
completed  in  about  45  minutes  using  the  "PCR"  algorithm  common  to  both  assays  evaluated  in  this 
study. 


Unlike  the  majority  of  systems  tested  in  this  evaluation,  the  T-COR  4  does  not  have  integrated  sample 
preparation  or  nucleic  acid  extraction.  Therefore,  purified  DNA  rather  than  whole,  inactivated  agent, 
was  used  in  the  assessment  of  the  device.  The  T-COR  4  is  simple  to  set  up  and  operate  and  consists  of  a 
base  unit  capable  of  analyzing  up  to  four  samples  or  controls  simultaneously  and  a  separate  mini¬ 
centrifuge  with  custom  rotor.  Both  the  base  unit  and  mini-centrifuge  are  powered  by  integrated, 
rechargeable  batteries.  Each  kit  is  supplied  with  four  assay  tubes  containing  lyophilized  reagents  within 
a  sealed  package.  To  analyze  a  sample  on  the  device,  the  user  rehydrates  tubes  with  27pL  water,  adds 
3pL  sample  or  control,  closes  the  tubes,  and  mixes  the  reactants.  Just  prior  to  loading  the  tubes  into  the 
device,  the  user  spins  them  briefly  with  the  mini-centrifuge  to  collect  the  reaction  components  in  the 
bottom  of  the  tube.  The  user  selects  the  desired  program  from  a  list  of  loaded  assays  using  the  four  hard 
keys  on  the  device  and  selects  start.  Results  are  viewable  in  real-time  as  a  fluorescence  versus  cycle 
graph.  Additionally,  a  table  displaying  CT  values  can  be  viewed. 


No  multiplexing  was  performed  during  this  evaluation,  as  the  system  is  comprised  of  four  two-color  real 
time  PCR  modules  for  a  single  analyte  detection  and  internal  positive  control  per  module.  As  an 
alternative  to  multiplex  assays,  a  user  can  run  four  separate  assays  simultaneously  as  long  as  each  assay 
requires  identical  thermal  cycling  parameters.  All  testing  was  performed  with  DNA  purified  from 
bacterial  reference  materials  using  the  Qiagen*  QIAamp*  DNA  Mini  Kit. 
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Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Purified  DNA  extracted  from  gamma-irradiated  Bacillus  anthracis  spores  was  serially  diluted  in  10-fold 
increments  into  water  and  evaluated  on  the  T-COR  4  platform  with  the  B.  anthracis  pOX-1  assay.  Each 
dilution  was  tested  in  triplicate  until  a  dilution  was  determined  to  yield  inconsistent  or  no  detection  of  B. 
anthracis.  SmartCycler  tubes  containing  lyophilized  reagents  were  rehydrated  with  27pL  water  then  3pL 
sample  was  added.  As  directed  by  kit  instructions,  samples  were  analyzed  using  the  preinstalled  PCR 
Program  algorithm.  The  LOD  of  the  device  was  determined  to  be  2. 38X101  CFU/mL. 


Table  25.  Bacillus  anthracis  LOD 


Concentration 

(femtograms 

(fgWL) 

Total  fg 

Total  Genome 
Equivalents  (GE) 

Equivalent 

CFU/mL 

Results  by  Loci 
(Positives/Total  Runs) 

pXOl 

3.20x10 1 

9.60  xlO 1 

1.70x10 1 

2.38X101 

3/3 

3.20x10° 

9.60  xl0° 

1.70x10° 

2.38xl02 

3/3 

3.20X101 

9.60  xlO1 

1.70X101 

2.38x10s 

3/3 

Vendor  Claimed  LOD:  N/A 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Purified  DNA  extracted  from  gamma-irradiated  Yersinia  pestis  cells  was  serially  diluted  in  10-fold 
increments  into  water  and  evaluated  on  the  T-COR  4  platform  with  the  Y.  pestis  assay.  The  actual  target 
of  the  assay  is  not  stated  by  Tetracore.  Each  dilution  was  tested  in  triplicate  until  a  dilution  was 
determined  to  yield  inconsistent  or  no  detection  of  B.  anthracis.  SmartCycler  tubes  containing 
lyophilized  reagents  were  rehydrated  with  27pL  water  then  3pL  sample  was  added.  As  directed  by  kit 
instructions,  samples  were  analyzed  using  the  preinstalled  PCR  program  algorithm.  The  LOD  of  the 
device  was  determined  to  be  1.80xl03CFU/mL. 


Table  26.  Yersinia  pestis  LOD 


Concentration 
(fg/  M-L) 

Total  fg 

Total  GE 

Equivalent 

CFU/mL 

Results  by  Loci 
(Positives/Total  Runs) 

Yersinia  murine  toxin  (ymt) 

1.30x10° 

3.91x10° 

7.51x10  1 

1.80xl02 

2/3 

1.30X101 

3.91X101 

7.51x10° 

1.80xl03 

3/3 

1.30xl02 

3.91xl02 

7.51X101 

1.80xl04 

3/3 

Vendor  Claimed  LOD:  N/A 


Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 
No  Assay  Available 
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J^Tetracore® 

Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

No  Assay  Available 

Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

Tetracore  supplies  a  C.  botulinum  A/B  assay  for  the  T-COR  4.  Although  the  target  is  DNA  rather  than  the 
active  toxin  used  in  this  assessment,  the  assay  was  evaluated  for  its  ability  to  detect  residual  host  DNA  in 
the  toxin  preparation.  BoNT  A,  as  supplied  by  Metabiologics,  is  produced  in  its  native  organism; 
therefore,  the  toxin  preparation  was  tested  to  determine  if  it  contained  detectable  C.  botulinum  DNA. 
Samples  were  tested  in  duplicate  along  with  a  single  replicate  each  of  a  positive  and  negative  control. 
One  of  the  two  replicates  at  l,000ng/mL  was  reported  as  positive  for  C.  botulinum  DNA,  but  the  form  of 
the  "fluorescence  versus  cycle"  plot  was  not  shaped  as  expected  for  typical  qPCR  data.  Therefore,  this 
positive  detection  was  discounted.  Lack  of  detectable  levels  of  C.  botulinum  DNA  in  the  toxin 
preparation  is  consistent  with  information  obtained  from  Metabiologics  that  the  product  had  very  low 
A26o  measurement,  indicative  of  no  host  DNA. 


Table  27.  Clostridium  botulinum  toxin  Limit  of  Detection  LOD 


Concentration 
(nanograms  (ng)/mL) 

Total  Toxin  (ng) 

Instrument  Call 

Results  by  Loci 
(Positives/Total  Runs) 

Unknown 

l.OOxlO2 

3.00X101 

0/2 

0/2 

1.00xl03t 

3.00x10° 

1/2 

1/2 

1  Positive  sample  had  a  very  low  CT  (12.55)  and  abnormal  fluorescence  versus  cycle  curve  shape 
Vendor  Claimed  LOD:  N/A 


Discussion 
Call  Assignments 

Call  assignments  are  made  based  upon  two-color  real-time  PCR  reactions.  The  internal  positive  control 
assures  amplification  and  lack  of  significant  PCR  inhibition.  The  control  was  successful  both  in  no 
template  controls  and  all  concentrations  of  analytes  tested.  The  second  color  channel  indicates  signal 
from  the  target  analyte  amplification.  A  proprietary  algorithm  provides  analysis  of  the  exponential  signal 
curve,  providing  a  "Smart  CT"  value  and  evaluating  curve  fitness.  Results  are  either  "pos"  or  "neg"  with  a 
given  CT  value  if  present.  A  "neg"  result  requires  a  positive  internal  control  result  to  rule  out  a  false 
negative  reaction. 

Plasmid  and  Chromosomal  Targets 

The  B.  anthracis  assay  tested  a  PCR  target  on  the  pXOl  plasmid,  which  is  usually  present  in  multiple 
copies.  This  has  both  advantages  and  disadvantages  as  a  detection  strategy,  as  higher  copy  number 
allows  for  higher  confidence  detection  and  lower  LOD.  However,  relying  on  a  single  plasmid  for  positive 
detection  of  B.  anthracis  is  known  to  have  significant  specificity  problems  as  other  members  (both 
pathogenic  and  non-pathogenic)  of  closely  related  members  of  the  Group  I  Bacilli,  and  especially  B. 
cereus,  often  share  partial  or  entire  plasmid  sequences  in  pXOl  and  pX02.  This  approach  of  assay  choice 
and  instrument  configuration  has  advantages  such  as  the  ability  to  determine  extent  of  contamination 
following  a  known  release  of  agent,  but  would  require  high-confidence  confirmation  before  taking  high- 
consequence  actions  in  response  to  a  positive  result. 
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Tetraeore3 


The  target  of  the  Y.  pestis  assay  is  ymt  on  pMTl  and  the  sensitivity  is  on  close-order  to  the  results  of  the 
low-copy  plasmid  targets.  The  small,  high-copy  pPCPl  plasmid  would  be  expected  to  more  sensitive.  Any 
advantage  here  would  be  more  obvious  when  looking  at  sampling  efficiency  and  the  ability  to  detect 
trace  amounts  of  larger  samples  (e.g.,  swabs)  following  sample  processing  into  a  small  volume  of 
template. 

Assay  Sensitivity 

Unlike  the  majority  of  other  systems  tested  in  this  study,  the  T-COR  4  system  required  purification  of  the 
DNA  prior  to  sample  analysis.  As  such  the  LOD  determined  in  this  study  was  extrapolated  using  the 
quality  control  metrics  supplied  by  the  vendor  and  an  approximate  genome  size  to  normalize  the  results 
and  adjust  the  units  for  easier  comparison  to  other  devices.  The  values  reported  here  do  not  take  in  to 
account  any  inefficiencies  in  the  DNA  extraction  procedure  nor  correct  for  variability  in  copy  number  of 
the  plasmid  targets. 

Assay  sensitivity  was  demonstrated  to  be  as  low  as  24  CFU/mL  with  100%  detection  of  B.  anthracis 
which  is  excellent  considering  the  sample  input  is  only  3pL  of  purified  DNA.  Sensitivity  of  the  Y.  pestis 
assay  was  approximately  10-fold  higher  reflecting  variability  in  target  sensitivity. 

Like  the  other  PCR  systems  in  this  study,  the  T-COR  4  was  unable  to  reliably  detect  trace  amounts  of 
Botulinum  DNA  in  the  toxin  preparation.  One  replicate  was  reported  as  positive  although  curve  shape 
was  not  indicative  of  a  true  amplification.  This  result  highlights  a  limitation  of  the  device  -  the  lack  of 
curve  fit  metrics  for  determining  CTs. 
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NIDS® 


by  ANP  Technologies 


System  Specifications 


Vendor:  ANP  Technologies 
Website:  www.anptinc.com 

System  Cost:  $6,500.00 
Assay  Cost:  $45.00 

Assay  Storage  Requirements:  Room  Temperature 

Agents  Tested  per  Assay:  5  per  Assay,  Multiple  formats  available 

Assay  Shelf  Life:  2  years  from  receipt  at  room  temperature 

Sample  Size  Required:  100-200  pL 

Type  of  Detection:  Antibody 

Time  to  Result:  15  minutes 

System  Weight:  1.6  lbs 

Operating  Range:  40-122  °F  (4-50  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  NIDS  assay  used  with  Stand  Alone  Reader  had 
successful  operation  in  Mobile  lab  and  field. 


System  Description 


ANP  has  developed  the  Nano  Intelligent  Detection  System, 
or  NIDS.  The  NIDS  is  a  multiplexed  Handheld  Assay  (HHA) 
together  with  a  palm-sized,  portable,  ruggedized  optical 
Stand  Alone  Reader  (SAR  III).  The  NIDS  technology  uses  an 
antibody  nanomanipulation  technique  that  orients  each 
antibody  so  that  there  are  optimal  biosensing  regions 
available  for  antigen  binding  and  sandwich  formation. 
Because  of  this  nanomanipulation,  the  NIDS  is  the  first 
HHA  that  claims  to  have  no  “Hook  Effect”.  The  NIDS  was 
designed  to  take  all  of  the  guess  work  out  of  interpreting 
HHA  results  by  different  responders  in  field  conditions  with 
poor  lighting  and  limited  visibility  while  wearing  personal 
protective  equipment. 


Test  Bed  Review 


The  NIDS  is  small,  rapid  and  has  the  ability  to  detect 
bacterial,  viral,  and  protein  toxins.  However,  it  still  delivers 
below  average  sensitivity  and  high  false  positive  rates  that 
we’ve  come  to  expect  with  a  typical  HHA.  Detection  levels 
ranging  from  lxlO6  to  lxlO8  CFU/mL  for  a  singleplex  assay 
and  lxlO6  to  lxlO9  CFU/mL  in  the  multiplex,  gave  the  NIDS 
an  overall  low  score  in  the  Target  Identification  Category  of 
the  Criteria  Table. 

Where  the  NIDS  did  fair  well  is  ease  of  use  and  small  size. 
Little  to  no  sample  preparation  time  is  required  for  this 
system,  maintenance  time  is  low  and  the  overall  sample  to 
result  is  15  minutes.  These  attributes  yielded  scores  of 
70-100%  in  4  of  the  5  categories. 

In  the  field,  the  end-users  were  impressed  with  the  ease 
of  set-up  and  minimum  amount  of  training  required. 
However  one  operator,  familiar  with  typical  HHAs  remarked 
“The  reader  makes  it  easier  than  current  HHAs,  but  it’s 
still  an  HHA.”  In  addition,  one  user  experienced  false 
positives  while  running  the  device,  which  was  quickly  fixed 
after  cleaning  the  lens.  However  this  result  did  lead  to 
concerns  of  contamination.  On  a  positive  note,  one  soldier 
commented,  “The  device  is  small  enough  to  fit  into  the 
pouch  of  my  body  armor”. 

As  a  result  of  testing  in  an  analytical  and  mobile 
laboratory  and  in  the  ECBC  Skippers  Point  Site  the  NIDS 
overall  performance  was  mediocre  at  best.  As  one  of 
the  only  currently  fielded  systems  in  our  test  bed,  better 
results  were  expected  from  this  system  and  users  were 
disappointed  by  the  recurring  false  positives.  Clearly  there 
is  a  need  for  a  more  sensitive  and  accurate  identification  of 
biothreats 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


|plNGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

l.OOxlO6 

O 

l.OOxlO7 

Yersinia  pestis 

2.50x10s 

O 

2.50xl06 

Vaccinia 

l.OOxlO6 

O 

>1.00xl08 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

O 

l.OOxlO9 

Clostridium 

Botulinum  Toxin 

5-OOxlO1 

© 

5-OOxlO1 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO7 

© 

l.OOxlO7 

Yersinia  pestis 

l.OOxlO6 

© 

l.OOxlO6 

Vaccinia 

>1.00xl08 

© 

l.OOxlO8 

Venezuelan  Equine 
Encephalitis 

l.OOxlO9 

© 

l.OOxlO9 

Clostridium 

Botulinum  Toxin 

5-OOxlO1 

O 

5.00xl02 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 


ANP  Technologies'  Nano  Intelligent  Detection  System  (NIDS)  is  a 
multiplexed  Handheld  Assay  (HHA)  together  with  a  palm-sized,  portable, 
ruggedized  optical  Stand  Alone  Reader  (SAR  III).  The  NIDS  technology 
uses  an  antibody  nanomanipulation  technique  that  orients  each 
antibody  so  that  there  are  optimal  biosensing  regions  available  for 
antigen  binding  and  sandwich  formation.  Because  of  this 
nanomanipulation,  the  NIDS  is  the  first  HHA  that  claims  to  have  no 
"hook  effect,"  which  causes  false  negative  results  at  high  target 
concentrations  The  NIDS  was  designed  to  take  the  guess  work  and 
inconsistency  out  of  interpreting  HHA  results  that  is  sometimes 
experienced  by  different  end-users  wearing  personal  protective 
equipment  in  field  conditions  with  poor  visibility. 

NIDS  assays  are  targeted  against  multiple  agents  -  up  to  5  -  in  the  same  assay,  in  contrast  to  other 
hand-held  lateral  flow  assays  that  have  single  targets  per  assay  cartridge.  By  reducing  the  "hook  effect," 
the  NIDS  design  also  reduces  the  need  to  run  serial  dilutions  to  obtain  reliable  results.  The  NIDS  assays 
offer  the  ability  to  directly  detect  proteins  from  bacteria,  viruses,  or  protein-based  toxins  and  to  provide 
an  orthogonal  confirmatory  technology  to  nucleic  acid-based  systems.  Other  desirable  characteristics  of 
this  system  are  the  low  cost,  small  size,  battery  power,  two  year  shelf-life,  and  simplicity  of  use.  ANP  has 
offered  various  NIDS  multiplex  assays  for  combinations  of  biological  threat  targets  in  2-plex,  3-plex,  4- 
plex  and  5-plex  configurations. 

The  NIDS  assays  are  simple  to  operate,  requiring  the  user  to  directly  add  lOOpL  of  agent  to  the  sample 
pad.  End-users  have  flexibility  in  adding  samples  to  the  NIDS  via  either  a  pipette  or  a  dropper  or  other 
disposable  device.  After  a  15  minute  development  time,  the  cartridges  are  inserted  into  the  SAR  III.  With 
very  few  manipulations  of  the  SAR  III  buttons,  the  detection  data  and  an  image  of  the  biosensing  region 
of  the  NIDS  are  stored  for  subsequent  review.  The  SAR  III  is  supplied  with  a  microSD  card  that  stores  up 
to  300  cassette  images  with  associated  sample  data.  The  SAR  III  can  obtain  power  from  either  two 
double-A  batteries,  a  USB  power  cord,  or  a  direct  110  volt  power  cord. 

The  NIDS  with  SAR  III  system  was  tested  using  ANP's  commercially  available  5-plex  Biothreat  Assays  to 
assess  the  technology  and  verify  vendor  claims  of  LOD  for  five  biothreat  agents  (two  bacterial,  two  viral, 
and  one  protein  toxin).  The  "5-Plex  1"  cartridge  was  designed  to  detect  Bacillus  anthracis,  Francisella 
tularensis,  Ricin,  Staphylococcal  enterotoxin  B,  and  BoNT  A.  The  "5-Plex  2"  cartridge  was  designed  to 
detect  Yersinia  pestis,  Orthopox  Vaccinia,  Venezuelan  Equine  Encephalitis,  Coxiella  Burnetii,  and  Brucella 
melitensis.  For  this  testing,  B.  anthracis  and  BoNT  A  were  assessed  using  "5-Plex  1"  and  Y.  pestis,  VAC 
and  VEE  were  evaluated  using  the  "5-Plex  2"  cartridges. 

Data 

Bacillus  anthracis 

Gram  positive  spore-forming  bacteria 

Gamma-irradiated  B.  anthracis  spores  were  diluted  to  1x10s,  lxlO6,  and  lxlO7  CFU/mL  in  assay  buffer 
(PBS,  0.1%  Triton  X-100,  0.04%  Kathon*  CG/ICP)  supplied  by  ANP  then  tested  on  "5-Plex  1"  cartridges. 
One  hundred  microliters  of  each  sample  was  applied  to  the  sample  pad  in  triplicate  assays  and  allowed 

92 


Approved  for  Unlimited  Distribution 
Unclassified 


ANTIBODYTECHNOLOGIES 


to  develop  for  15  minutes.  The  cartridges  were  imaged  on  the  SAR  III.  Although  the  vendor  claimed  the 
LOD  to  be  lxlO6  CFU/mL,  10-fold  more  was  required  for  successful  detection.  The  LOD  of  the  assay  was 
determined  to  be  l.OOxlO7  CFU/mL. 


NOTE:  A  single  false  positive  identification  of  F.  tulorensis  occurred  during  assays  of  the  lowest 
concentrated  samples.  However,  the  result  could  not  be  repeated  by  additional  imaging  of  the  cassette. 
Visual  examination  of  the  lens  surface  after  obtaining  the  false  positive  found  the  glass  surface  to  be 
clean  and  without  smudges. 


Table  28.  Bacillus  anthracis  LOD 


Concentration 
(Colony  Forming 
Units  (CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents  (GE) 

Results 

(Positives/Total  Runs) 

1.00x10s* 

l.OOxlO4 

2.38xl04 

0/3 

1.00x10s 

1.00x10s 

2.38xl05 

0/3 

l.OOxlO7 

1.00x10® 

2.38x10® 

3/3 

*  Single  false  positive  result  for  F.  tularensis 
Vendor  Claimed  LOD:  1.00x10s  CFU/mL 


Yersinia  pestis 

Gram  negative  bacteria 

Gamma-irradiated  Y. pestis  was  diluted  to  2.5xl04,  2.5x10s,  and  2.5xl06  CFU/mL  in  assay  buffer  supplied 
by  ANP  (PBS,  0.1%  Triton  X-100,  0.04%  Kathon®  CG/ICP)  then  tested  on  the  "5-Plex  2"  cartridge.  One 
hundred  microliters  of  each  sample  was  applied  to  the  sample  pad  in  triplicate  and  allowed  to  develop 
for  15  minutes.  After  the  development  period,  the  cartridges  were  imaged  on  the  reader.  Although  the 
vendor  claimed  LOD  of  the  assay  to  be  2.5x10s  CFU/mL,  10-fold  more  /.  pestis  was  required  for 
successful  detection.  The  LOD  of  the  assay  was  determined  to  be  2.50xl06  CFU/mL. 


Table  29.  Yersinia  pestis  LOD 


Concentration 

Total  CFU 

Total  GE 

Results 

(CFU/mL) 

(Positives/Total  Runs) 

2.5xl04 

2.5xl03 

3.46xl03 

0/3 

2.5x10s 

2.5xl04 

3.46xl04 

0/3 

2.5x10® 

2.5x10s 

3.46x10s 

3/3 

Vendor  claimed  LOD:  2.50x10s  CFU/mL 


Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  virus  was  diluted  to  lxlO4,  1x10s,  and  lxlO6  PFU/mL  in  assay  buffer  supplied  by 
ANP  (PBS,  0.1%  Triton  X-100,  0.04%  Kathon®  CG/ICP)  then  tested  on  the  "5-Plex  2"  cartridge.  One 
hundred  microliters  of  each  sample  was  applied  to  the  sample  pad  in  triplicate  and  allowed  to  develop 
for  15  minutes.  After  the  development  period,  the  cartridges  were  imaged  on  the  reader.  None  of  the 
nine  samples  tested  resulted  in  a  positive  detection  of  VAC;  therefore,  a  single  sample  at  1x10s  PFU/mL 
was  tested  and  also  found  to  be  negative.  Testing  was  repeated  with  samples  diluted  to  lxlO7  and  1x10s 
PFU/mL.  In  the  repeated  testing,  a  single  replicate  of  the  highest  concentration  tested  yielded  a  positive 
result  (see  Table  30).  Therefore,  the  LOD  of  the  assay  for  VAC  is  greater  than  1x10s  PFU/mL.  Testing  of 
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the  antigen  at  higher  concentration  would  have  required  the  sample  to  be  added  without  dilution; 
therefore,  it  was  not  completed. 


Table  30.  Vaccinia  LOD 


Concentration 

PFU 

Total  GE 

Results 

(PFU/mL) 

(Positives/Total  Runs) 

1.00x10s 

l.OOxlO4 

1.44xl04 

0/3 

l.OOxlO6 

1.00x10s 

1.44x10s 

0/3 

l.OOxlO7 

1.00x10s 

1.44xl06 

0/6 

1.00x10s 

l.OOxlO7 

1.44xl07 

1/4 

Vendor  Claimed  LOD:  N/A 


Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

Gamma-irradiated  VEE  virus  was  diluted  to  1x10s  and  lxlO9  PFU/mL  in  assay  buffer  supplied  by  ANP 
(PBS,  0.1%  Triton  X-100,  0.04%  Kathon®  CG/ICP)  then  tested  on  the  "5-Plex  2"  cartridge.  One  hundred 
microliters  of  each  sample  was  applied  to  the  sample  pad  in  triplicate  and  allowed  to  develop  for  15 
minutes.  Although  the  vendor  claimed  LOD  of  the  assay  to  be  lxlO8  PFU/mL,  10-fold  more  was  required 
for  successful  detection  in  this  study.  The  LOD  of  the  assay  was  determined  to  be  lxlO9  PFU/mL. 


Table  31.  Venezuelan  Equine  Encephalitis  LOD 


Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

lxlO8 

lxlO7 

lxlO7 

0/3 

lxlO9 

1x10s 

1x10s 

3/3 

Vendor  Claimed  LOD  l.OOxlO8  PFU/mL 


Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

BoNT  A  was  diluted  to  five,  50,  and  500ng/mL  into  PBS  then  tested  on  the  "5-Plex  1"  assay  cartridge. 
One  hundred  microliters  of  each  sample  was  applied  to  the  sample  pad  in  triplicate  and  allowed  to 
develop  for  15  minutes.  After  the  development  period,  the  cartridges  were  imaged.  The  vendor  claimed 
LOD  of  the  assay  is  50ng/mL;  this  limit  was  confirmed  during  this  testing. 

Table  32.  Clostridium  botulinum  Type  A  LOD _ 


Concentration 

(ng/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

5x10° 

5.00x10  1 

0/3 

SxlO1 

5.00x10° 

3/3 

5xl02 

5-OOxlO1 

3/3 

Vendor  Claimed  LOD:  S.OOxlC^ng/mL 


Multiplex  Testing 

All  five  antigens  used  in  this  assessment  can  be  detected  by  the  NIDS  cartridges,  but  the  targets  are 
spread  over  two  cartridges,  5-Plex  1  and  5-Plex  2.  For  multiplex  testing,  samples  were  created 
containing  only  antigens  that  could  be  detected  with  the  assay  used.  Triplicate  samples  containing 
antigens  diluted  to  the  concentration  determined  as  LOD  through  empiric  singleplex  testing  were 
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evaluated  by  applying  lOOpL  to  the  appropriate  cartridge.  After  the  development  period,  the  cartridges 
were  imaged.  The  singleplex  LOD  for  each  assay  was  confirmed  as  valid  for  multiplex  testing  except 
BoNT  A.  The  B.  onthrocis/ BoNT  A  evaluation  was  repeated  with  the  concentration  of  the  toxin  increased 
10-fold.  Both  targets  were  detected  in  all  three  replicates  upon  repeat  testing. 


Table  33.  5-Plex  1  Multiplex  Evaluation 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

B.  anthracis 

l.OOxlO7* 

1.00x10s 

2.38x10s 

6/6 

Agent 

Concentration 

(ng/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

BoNT  A 

5-OOxlO1 

5.00x10° 

1/3 

5.00xl02 

5-OOxlO1 

3/3 

Table  34.  5-Plex  2  Multiplex  Evaluation 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

Y.  pestis 

2.50x106 

2.50x10s 

3.46x10s 

3/3 

Agent 

Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

VAC 

1.00x10s 

l.OOxlO7 

1.44xl07 

3/3 

VEE 

l.OOxlO9 

l.OOxlO8 

l.OOxlO8 

3/3 

*  Single  false  positive  result  for  F.  tularensis 


Discussion 
Call  Assignment 

Call  assignments  are  made  based  upon  SAR  III  optical  scanning  of  five  sectors  of  the  NIDS  assay 
cartridge.  The  use  of  the  scanner  and  software  band-calling  software  eliminates  the  subjectivity  of 
manual  result  assignments,  and  is  intended  for  consistent  detection  of  faint  results.  However,  the 
laboratory  assessment  discovered  that  the  software  occasionally  called  positive  results  where  no  band 
was  apparent  (i.e.,  false  positive).  Each  cartridge  includes  an  assay  positive  control  to  ensure  fluidics, 
assay  chemistry,  and  detection  hardware  is  functioning  properly.  Data  is  displayed  as  "OK"  (positive 
control  only),  red  (positive),  and  green  (negative)  for  each  assay  region.  Unfortunately,  the  software 
makes  a  call  by  a  region  rather  than  assay  target  which  may  occasionally  lead  to  confusion  for  the  end 
user. 

Assay  Sensitivity 

Assay  sensitivity  across  bacterial,  viral,  and  toxin  targets  was  relatively  poor,  in  comparison  to  nucleic 
acid-based  detection  systems  and  even  other  antibody-based  systems.  In  most  cases  (except  for  the 
protein  toxin),  the  LOD  was  one  to  two  orders  of  magnitude  less  sensitive  than  vendor  claims.  For  all 
organisms,  the  LOD  was  at  or  above  the  LD50  or  ID50  of  the  organism  or  toxin.  Some  testing  was  limited 
by  the  availability  of  high  enough  concentrations  of  stock  target  solution.  A  major  implication  is  that  the 
NIDS  may  not  have  true  utility  for  biological  agent  identification  except  in  cases  where  gross 
contamination  is  present,  the  inference  being  that  the  system  will  produce  false-negative  results  when 
only  modest  (but  still  dangerous)  amounts  of  agent  are  present. 

In  singleplex  testing,  a  single  false-positive  result  was  noted  (Francisella  tularensis)  as  positive  when  a 
small,  undetectable  amount  of  B.  anthracis  was  tested.  Rescanning  of  the  assay  cartridge  with  this  false- 
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positive  result  was  consistently  positive  but  the  result  could  not  be  recapitulated  with  any  amount  of  B. 
anthracis  antigen.  Another  false-positive  F.  tulorensis  result  was  seen  in  the  multiplex  B.  anthracis  and 
BoNT  A  testing,  but  the  result  was  not  seen  with  repeated  scans  of  the  same  cartridge.  Contamination 
with  F.  tulorensis  antigen  is  unlikely,  given  the  high  concentrations  of  materials  that  would  be  required 
to  produce  the  result  and  the  fact  that  this  antigen  was  not  in  use  in  the  laboratory  at  any  time  during 
instrumentation  evaluation.  It  is  possible  that  the  assay  itself  has  an  inherent  cross  reactivity  that  will 
need  to  be  addressed  to  improve  specificity.  Another  possible  explanation  is  that  fluctuations  of 
background  intensities  in  the  sector  with  the  F.  tulorensis  antibodies  were  sufficient  for  the  SAR  III 
software  algorithm  to  give  false-positive  calls. 

The  BoNT  A  assay  had  almost  an  order  of  magnitude  reduction  of  sensitivity  in  multiplex  testing 
compared  to  the  LOD  seen  in  singleplex  assays.  This  result  was  unexpected,  as  multiplex  immunoassays 
are  not  subject  to  reaction  kinetics  and  limiting  reagent  problems  inherent  in  PCR-based  systems  where 
consistently  sensitive  multiplex  detection  is  challenging. 

A  few  technical  issues  also  limited  sensitivity,  requiring  some  initial  trouble-shooting  before  the  formal 
test  plan  could  be  executed.  First,  a  common  detergent  frequently  used  in  sample  buffers  for 
immunoassay  (Tween-20)  was  later  confirmed  by  the  vendor  to  be  inhibitory  to  the  assay  system.  This 
required  the  use  of  a  proprietary,  vendor-supplied  buffer.  In  addition,  there  was  some  ambiguity  as  to 
the  age  and  shelf  life  of  the  vendor-provided  cartridges,  calling  into  question  the  validity  of  the  initial 
data  demonstrating  poor  sensitivity.  This  required  additional  testing  of  multiple  lots  of  assay  cartridges. 
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MagPix™ 


by  Luminex,  Corp 


System  Specifications 


Vendor:  Luminex,  Corp. 

Website:  www.luminexcorp.com 

System  Cost:  $24,000.00 

Assay  Cost:  Price  on  Request 

Assay  Storage  Requirements:  Refrigeration 

Agents  Tested  per  Assay:  Up  to  50 

Assay  Shelf  Life:  User  determined 

Sample  Size  Required:  20-200  pL 

Type  of  Detection:  Antibody 

Time  to  Result:  90-105  minutes  for  No  Wash  Assay 

System  Weight:  38.5  lbs 

Operating  Range:  50-104  °F  (10-40  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S. 

DoD’s  ASD  (R&E).  Assay  integration  requires  development  and 
optimization. 

*  ^-7  g  aj 


System  Description 


The  Luminex  MAGPIX  utilizes  labeled  magnetic  beads  in 
a  fluidics  system  combined  with  optical  detection  and 
computerized  analysis  to  perform  plate-based  multiplex 
immunoassays.  Preparation  of  the  sample(s)  is  done  in 
a  96-well  plate  on  the  benchtop  or  in  a  biological  safety 
cabinet  and  consists  of  multiple  incubations  with  antibody- 
and  fluorescently-labeled  magnetic  beads.  The  operator 
sets  assay  parameters  in  xPONENT  software  on  a  computer 
that  controls  the  MAGPIX  and  loads  the  assay  plate  to  the 
MAGPIX,  which  runs  the  analysis  of  the  prepared  samples. 
The  MAGPIX  can  run  highly  multiplexed  assays  to  measure 
up  to  50  different  analytes  simultaneously  in  a  sample. 

The  MAGPIX  also  has  capability  to  perform  nucleic  acid 
detection  utilizing  hybridizations  to  nucleic  acid-labeled 
magnetic  beads  and  has  developed  a  prototype  sample 
preparation  cartridge  that  would  automate  sample 
preparation.  For  this  testing,  all  assays  were  developed  by 
the  laboratory  scientists  at  ECBC. 


Test  Bed  Review 


MAGPIX  had  one  of  the  largest  footprints  of  the  biological 
identifiers,  including  computer,  magnetic  bead  separator 
and  a  plate  shaker.  Despite  the  utilization  of  a  faster 
“no-wash”  ELISA-type  assay  format,  the  assays  still  took 
approximately  2  hours.  The  software  required  training 
and  practice  to  become  comfortable  at  running  assays 
and  analyzing  the  data.  MAGPIX  assays  utilized  the  same 
antibodies  as  other  systems  in  this  assessment,  yet  were 
more  sensitive  than  most  other  devices.  Limits  of  detection 
were  approximately  105  CFU/  mL  sample  for  bacteria 
assays,  107  PFU/mL  sample  for  virus  assays  and  1  ug/mL 
for  toxin  assays. 

Because  of  its  size,  the  MAGPIX  was  assessed  in  the 
Mobile  Laboratory,  but  not  the  field  setting.  Mobile  lab 
end-users  gave  mostly  split  opinions  (among  FAIR,  GOOD 
and  EXCELLENT  scores)  on  the  MAGPIX.  The  exception  was 
for  “Ease  of  Use”  which  rated  FAIR,  since  this  instrument 
requires  some  training  and  more  sample  preparation 
than  most  other  biological  agent  identifiers.  One  operator 
with  flow  cytometry  experience  was  comfortable  with  the 
MAGPIX  and  affirmed,  “The  MAGPIX  is  similar  to  other 
Luminex  devices.”  The  perceived  safety  was  ranked  either 
FAIR  or  GOOD  due  to  operators  concerns  over  handling 
the  96-well  plates  with  possible  pathogens.  Users  felt  they 
could  minimize  the  safety  concerns  by  extensive  use  of 
bleach  and  the  biological  safety  cabinet  within  the  Mobile 
laboratory.  The  operators  suggested,  “The  MAGPIX  should 
be  used  as  a  confirmatory  technology,  after  an  initial 
positive  PCR  assay.” 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

N/A 

O 

1.00x10s 

Yersinia  pestis 

N/A 

O 

1.00x10s 

Vaccinia 

N/A 

O 

l.OOxlO7 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

1.00x10s 

Clostridium 

Botulinum  Toxin 

N/A 

O 

l.OOxlO3 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO5 

O 

l.OOxlO7 

Yersinia  pestis 

l.OOxlO5 

O 

l.OOxlO6 

Vaccinia 

l.OOxlO7 

O 

>1.00xl08 

Venezuelan  Equine 
Encephalitis 

l.OOxlO8 

© 

l.OOxlO8 

Clostridium 

Botulinum  Toxin 

l.OOxlO3 

© 

l.OOxlO3 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


100  100 


U1 

Ln 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Luminex 


Overview 

The  Luminex  MAGPIX  utilizes  labeled  magnetic  beads  in  a  fluidics  system  combined  with  optical 
detection  and  computerized  analysis  to  perform  plate-based  multiplex  immunoassays.  Preparation  of 
the  sample(s)  is  done  in  a  96-well  plate  on  the  benchtop  or  in  a  biological  safety  cabinet  and  consists  of 
multiple  incubations  with  antibody-  and  fluorescently-labeled  magnetic  beads.  The  operator  sets  assay 
parameters  in  xPONENT  software  on  a  computer  that  controls  the  MAGPIX  and  loads  the  assay  plate  to 
the  MAGPIX,  which  runs  the  analysis  of  the  prepared  samples.  The  MAGPIX  can  run  highly  multiplexed 
assays  to  measure  up  to  50  different  analytes  simultaneously  in  a  sample.  The  MAGPIX  is  also  capable  of 
performing  nucleic  acid  detection  utilizing  hybridizations  to  nucleic  acid-labeled  magnetic  beads  and 
Luminex  has  developed  a  prototype  sample  preparation  cartridge  that  would  automate  sample 
preparation.  For  this  testing,  all  assays  were  developed  by  the  laboratory  scientists  at  ECBC. 

The  Luminex  MAGPIX  platform  running  multiplex  xMAP°  magnetic  bead-based  assays  with  existing  CRP 
antibody  pairs  was  tested  to  evaluate  the  technology  for  the  detection  of  five  biothreat  agents  (two 
bacterial,  two  viral,  and  one  protein  toxin).  Custom  assays  were  designed  by  coupling  a  capture, 
monoclonal  antibody  to  a  MagPlex  bead  via  a  carbodiimide  linkage  and  labeling  detection  antibodies 
with  biotin  via  an  N-hydroxysulfosuccinimide  bond  containing  a  22A  spacer.  Assays  were  optimized  to 
determine  the  best  concentrations  of  both  the  labeled  detector  antibodies  and  streptavidin-R- 
phycoerythrin  (SAPE)  conjugate.  Individual,  singleplex  assays  were  performed  to  determine  the  LOD  for 
each  assay  using  a  no-wash  format.  Briefly,  the  antibody  coupled  beads  were  blocked  with  1%  bovine 
serum  albumin  (BSA)  in  PBS  then  mixed  with  dilutions  of  target.  Detection  antibodies  were  added 
followed  by  the  SAPE  conjugate  with  30  minute  incubations  in  the  dark  at  ambient  temperature  with 
800  revolutions  per  minute  (rpm)  shaking  between  each  step.  The  volume  of  each  assay  component  was 
25pL;  therefore,  the  final  volume  of  the  assay  was  100pL.  Just  prior  to  analyzing  the  assay  on  the 
MAGPIX,  the  beads  were  trapped  with  a  magnet  and  washed  twice  with  PBS  +  0.1%  Tween-20  and 
resuspended  in  lOOpL  wash  buffer.  Seventy-five  pL  of  each  reaction  was  analyzed  and  the  average  mean 
fluorescent  intensity  (MFI)  was  recorded.  Time  to  result  is  approximately  two  hours  with  this  assay.  Each 
concentration  was  tested  in  triplicate  and  each  individual  sample  was  tested  in  triplicate  for  a  total  of 
nine  data  points  per  concentration.  The  MFI  of  the  individual  replicates  were  averaged  and  the  average 
MFI  was  compared  to  the  average  MFI  of  a  no-antigen  sample  (background).  Samples  were  considered 
positive  if  the  MFI  was  greater  than  three  times  the  average  background  MFI. 

The  xMAP  magnetic  bead  technology  has  completely  open  assay  architecture  with  flexible  surface 
covalent  conjugation  chemistry,  allowing  for  user-developed  immunoassay  (including  serology),  PCR, 
and  enzymatic  assay  configurations. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Gamma  irradiated  B.  anthracis  spores  were  diluted  to  l.OOxlO4,  1.00x10s,  and  l.OOxlO6  CFU/mL  into 
block  solution  (1%  BSA  in  PBS)  and  tested  in  triplicate  with  the  custom  B.  anthracis  assay.  Background  of 
the  B.  anthracis  assay  was  higher  than  the  other  four  custom  assays  but  was  only  approximately  20  MFI. 
The  detector  antibody  was  added  at  8pg/mL  while  the  SAPE  conjugate  was  added  at  16pg/mL.  Limit  of 
detection  of  the  assay  was  found  to  be  1.00x10s  CFU/mL  and  average  MFIs  ranged  116  to  155.  The  MFI 
of  all  nine  replicates  at  this  concentration  was  greater  than  60. 
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Table  35.  Bacillus  anthracis  LOD 


Concentration 
(Colony  Forming 
Units  (CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents  (GE) 

Results 

(Positives/Total  Runs) 

l.OOxlO4 

2.50xl02 

5.95xl02 

0/9 

1.00x10s 

2.50xl03 

5.95x10s 

9/9 

1.00x10s 

2.50xl04 

5.95xl04 

9/9 

Vendor  Claimed  LOD:  N/A 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma  irradiated  Y.  pestis  cells  were  diluted  to  l.OOxlO4,  l.OOxlO5,  and  l.OOxlO6  CFU/mL  into  block 
solution  (1%  BSA  in  PBS)  and  tested  in  triplicate  with  the  custom  Y.  pestis  assay.  Background  of  the  Y. 
pestis  assay  was  low,  approximately  13  MFI.  The  detector  antibody  was  added  at  8pg/mL  while  the  SAPE 
conjugate  was  added  at  16pg/mL.  Limit  of  detection  of  the  assay  was  found  to  be  l.OOxlO5  CFU/mL.  The 
average  MFIs  ranged  from  77  to  99.  The  MFI  of  all  nine  replicates  at  this  concentration  was  greater  than 
39. 


Table  36.  Yersinia  pestis  LOD 


Concentration 

Total  CFU 

Total  GE 

Results 

(CFU/mL) 

(Positives/Total  Runs) 

l.OOxlO4 

2.50xl02 

3.46xl02 

0/9 

l.OOxlO5 

2.50x10s 

3.46x10s 

9/9 

l.OOxlO6 

2.50xl04 

3.46xl04 

9/9 

Vendor  Claimed  LOD:  N/A 


Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma  irradiated  VAC  viral  particles  were  diluted  to  l.OOxlO6,  l.OOxlO7,  and  l.OOxlO8  PFU/mL  into 
block  solution  (1%  BSA  in  PBS)  and  tested  in  triplicate  with  the  custom  VAC  assay.  Background  of  the 
VAC  assay  was  low,  approximately  13  MFI.  The  detector  antibody  was  added  at  4pg/mL  while  the  SAPE 
conjugate  was  added  at  16pg/mL.  Limit  of  detection  of  the  assay  was  found  to  be  l.OOxlO7.  The  average 
MFIs  ranged  75  to  111.  The  MFI  of  all  nine  replicates  at  this  concentration  was  greater  than  39. 


Table  37.  Vaccinia  LOD 


Concentration 
(Plaque  Forming 
Units  (PFU)/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

l.OOxlO6 

2.50xl04 

3.61xl04 

0/9 

l.OOxlO7 

2.50x10s 

3.61xl05 

9/9 

1.00x10s 

2.50x10s 

3.61x10s 

9/9 

Vendor  Claimed  LOD:  N/A 


Venezuelan  Equine  Encephalitis 

+  sense  ssRNA,  Alphavirus 

Gamma  irradiated  VEE  viral  particles  were  diluted  to  l.OOxlO5,  l.OOxlO6,  l.OOxlO7  l.OOxlO8  and 
l.OOxlO9  PFU/mL  into  block  solution  (1%  BSA  in  PBS)  and  tested  in  triplicate  with  the  custom  VEE  assay. 
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Background  was  low,  approximately  12  MFI.  The  detector  antibody  was  added  at  2pg/mL  while  the  SAPE 
conjugate  was  added  at  16pg/mL.  Limit  of  detection  of  the  assay  was  found  to  be  1.00x10s  PFU/mL— 
average  MFIs  ranged  3241  to  9717.  The  MFI  of  all  nine  replicates  at  this  concentration  was  greater  than 
36.  There  was  a  steep  decline  in  intensity  for  this  assay  as  the  antigen  was  diluted  and  the  assay  has 
much  more  variance  in  the  individual  intensities  than  the  other  assays.  At  l.OOxlO7  PFU/mL,  for 
example,  there  is  a  100-fold  difference  in  the  intensities  of  some  of  the  individual  replicates.  Further 
development  and  optimization  of  this  assay  is  needed  to  enhance  the  performance  of  this  assay  but 
could  not  be  completed  within  the  scope  of  this  study. 


Table  38.  Venezuelan  Equine  Encephalitis  LOD 


Concentration 

Total  PFU 

Total  GE 

Results 

(PFU/mL) 

(Positives/Total  Runs) 

1.00x10s 

2.50xl03 

2.50xl03 

1/9 

l.OOxlO6 

2.50xl04 

2.50xl04 

4/9 

l.OOxlO7 

2.50x10s 

2.50x10s 

4/9 

l.OOxlO8 

2.50x10s 

2.50x10s 

9/9 

1.00x10® 

2.50xl07 

2.50xl07 

9/9 

Vendor  Claimed  LOD:  N/A 


Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

Active,  BoNT  A  complex  was  diluted  to  1.00x10°,  l.OOxlO1,  l.OOxlO2  and  1.00xl03ng/mL  into  PBS  and 
tested  in  triplicate  with  the  custom  BoNT  A  assay.  Background  for  the  assay  was  the  lowest  of  all  five 
assays  developed  -  approximately  8  MFI.  The  detector  antibody  was  added  at  4pg/mL  while  the  SAPE 
conjugate  was  added  at  8pg/mL.  Note,  the  SAPE  conjugate  was  added  at  half  concentration  in  this  assay 
relative  to  the  other  four  assays  which  likely  led  to  the  lowest  background.  The  LOD  of  the  assay  was 
found  to  be  l,000ng/mL.  The  average  MFIs  ranged  from  248  to  274.  The  MFI  of  all  nine  replicates  at  this 
concentration  was  greater  than  24. 

Table  39.  Clostridium  botulinum  toxin  Limit  of  Detection  LOD 


Concentration 
(nanograms  (ng)/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

1.00x10° 

0.025 

0/9 

l.OOxlO1 

0.25 

0/9 

l.OOxlO2 

2.5 

0/9 

l.OOxlO3 

25 

9/9 

Vendor  Claimed  LOD:  N/A 


Multiplex  Detection 

The  multiplex  assessment  of  the  five  targets  using  the  MAGPIX  demonstrated  critical  interferences 
between  the  assays.  For  the  multiplex  evaluation,  no  additional  optimization  was  performed  and  assays 
were  tested  with  the  same  conditions  as  had  been  developed  for  the  singleplex  detection.  The  exception 
was  the  BoNT  A  assay,  which  for  the  singleplex  required  8pg/mL  SAPE  conjugate  while  all  other  assays 
required  16pg/mL.  The  higher  amount  of  conjugate  was  used  for  the  assessment.  The  multiplex  assays 
were  not  fully  optimized  by  ECBC  staff  (as  is  normally  done  for  MAGPIX  multiplex  assays)  as  this  was 
outside  the  scope  of  the  testing.  The  results  of  this  study  may  have  been  more  favorable  if  multiplex 
assay  optimization  had  been  performed. 
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A  preliminary  multiplex  assessment  was  performed  without  VEE  and  BoNT  A.  For  that  assay,  triplicate 
samples  were  created  with  a  mixture  of  1.00x10s  CFU/mL  B.  anthracis,  1.00x10s  CFU/mL  Y.  pestis,  and 
l.OOxlO7  PFU/mL  VAC.  Each  sample  was  tested  in  triplicate  for  a  total  of  nine  individual  replicates  and 
the  replicates  for  each  sample  were  averaged.  Y.  pestis  was  detected  in  all  three  samples  while  B. 
anthracis  and  VAC  were  not. 

Multiplex  assessment  was  next  performed  to  detect  all  five  targets.  Triplicate  samples  were  created 
with  a  mixture  of  l.OOxlO6  CFU/mL  B.  anthracis ,  1.00x10s  CFU/mL  Y.  pestis ,  l.OOxlO8  PFU/mL  VAC, 
1.00x10s  PFU/mL  VEE,  and  1.00xl03ng/mL  BoNT  A.  Each  sample  was  tested  in  triplicate  for  a  total  of 
nine  replicates  and  the  intensity  values  for  each  sample  were  averaged.  B.  anthracis  was  not  detected  in 
any  sample  or  replicate.  Y.  pestis  and  VAC  were  each  detected  in  two  of  the  three  samples  and  seven  of 
the  nine  replicates.  VEE  was  detected  in  all  three  samples,  but  only  eight  of  the  nine  replicates,  while 
BoNT  A  was  detected  in  all  samples  and  replicates. 

The  concentrations  of  B.  anthracis  and  VEE  were  increased  10-fold  while  the  concentrations  of  the  other 
three  antigens  were  kept  the  same.  The  multiplex  assay  was  repeated  with  the  altered  sample.  VAC 
could  not  be  tested  at  a  higher  amount  due  to  the  concentration  of  the  stock  vial.  Detector  antibody  mix 
or  SAPE  conjugate  was  accidently  not  added  to  one  well,  which  caused  that  well  to  be  negative  for  all 
agents.  The  results  of  this  well  were  not  used  to  determine  sample  averages  nor  included  in  the 
individual  replicate  results.  B.  anthracis,  Y.  pestis,  VEE,  and  BoNT  A  were  detected  in  all  three  samples; 
VAC  was  not  detected  in  any.  With  the  exception  of  VAC,  for  which  no  replicates  were  positive,  all  eight 
replicates  were  positive  for  all  agents. 


Table  40.  Multiplex  Evaluation 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

B.  anthracis 

l.OOxlO5 

2.50xl03 

5.95xl03 

0/9 

1.00x10s 

2.50xl04 

5.95xl04 

0/9 

l.OOxlO7 

2.50x10s 

5.95x10s 

7/8 

Y.  pestis 

l.OOxlO5 

2.50xl03 

3.48xl03 

16/18 

1.00x10s 

2.50xl04 

3.48xl04 

8/8 

Agent 

Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

VAC 

l.OOxlO7 

2.50x10s 

3.60x10s 

0/9 

1.00x10s 

2.50x10s 

3.60x10s 

7/17 

VEE 

1.00x10s 

2.50x10s 

N/D 

8/9 

l.OOxlO9 

2.50xl07 

N/D 

8/8 

Agent 

Concentration 

(ng/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

BoNT  A 

l.OOxlO3 

2.50X101 

17/17 

Discussion 
Call  Assignments 

Individual  beads  are  bar-coded  with  two  dyes  (red  and  near  infra-red)  with  over  500  permutations,  50  of 
which  can  be  used  on  the  MAGPIX  platform.  Signal  is  generated  via  LED  excitation  of  phycoerythrin 
labeled  antibody/DNA  detection  probe.  Beads  are  imaged  via  CCD,  capturing  three-color  signatures 
simultaneously  to  de-multiplex  and  quantify  signal. 
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The  user  sets  the  threshold  for  positive  call  assignments,  usually  as  a  function  of  the  background  plus  a 
multiple  of  noise  or  standard  deviation  of  unbound  beads.  Analysis  is  performed  either  on  the  on-board 
software  package  or  exported  to  be  analyzed  externally.  Because  of  the  statistical  analysis  used  to  assign 
positive  and  negative  calls,  each  assay  is  usually  performed  in  triplicate.  The  data  above  is  presented  in 
aggregate,  with  three  assays  performed  in  triplicate  for  a  total  of  nine  data  points. 

Assay  Development 

Step  1:  Selection  of  antibody  pairs 

The  antibody  pairs  for  MAGPIX  assay  development  were  obtained  from  inventory  offered  by  the  CRP. 
The  particular  antibody  pair  for  each  target  had  previously  been  verified  by  the  CRP  to  effectively  detect 
the  target  in  sandwich-type  immunoassays.  The  individual  antibodies  that  were  selected,  with  CRP 
catalogue  identifier  and  production  number,  are  described  in  the  introduction. 

Step  2:  Coupling  of  capture  antibodies  to  MagPlex  beads 

The  monoclonal  capture  antibodies  were  coupled  to  MagPlex  microspheres  by  a  carbodiimide  reaction 
of  the  primary  antibody  amino  groups  to  the  carboxyl  functional  groups  on  the  microsphere  surface 
using  an  xMAP  Antibody  Coupling  Kit  at  5pg  antibody  per  lxlO6  microspheres.  Microspheres  were 
resuspended  to  5,000  beads  per  microliter  and  stored  at  4C  protected  from  light.  Coupling  was  assessed 
by  diluting  each  microsphere  to  50  beads  per  microliter  and  blocking  the  beads  overnight  in  a  PBS  +  1% 
BSA  solution.  The  labeled,  blocked  microspheres  were  then  reacted  with  titrated  anti-Mouse- 
Phycoerythrin  (PE)  and  analyzed  on  the  MAGPIX  instrument.  MFI  versus  concentration  of  anti-Mouse-PE 
was  plotted  and  the  curve  determined  to  visually  match  the  manufacturer's  expected  shape. 

Step  3:  Biotinylation  of  detector  antibodies 

The  detector  antibodies  were  biotinylated  using  a  Thermo  Fisher  EZ  Link  Sulfo-LC-NHS  Biotinylation  Kit 
at  290pg/mL  then  buffer  exchanged  into  PBS  using  2mL  Zeba  Spin  Desalting  Columns.  The  level  of 
biotinylation  was  measured  using  a  Thermo  Fisher  Biotin  Quantitation  Kit.  Incorporation  of  biotin  (mM 
Biotin  per  mM  Antibody)  was  measured  to  be  anti-B.  anthracis:  1.86;  anti-K  pestis:  7.33;  anti-VAC:  8.64; 
anti-VEE:  8.55;  and  anti-BoNT  A:  5.42.  Labeled  antibodies  were  stored  at  4°C. 

Step  4:  Checkerboard  Assay  Development  for  Optimization  of  Detector  and  SAPE  concentration 
A  checkerboard  assessment  was  performed  for  each  assay  to  determine  optimum  concentrations  of  the 
biotinylated  detector  antibody  and  SAPE  conjugate.  The  antibody  coupled  microspheres  were  diluted  to 
100  beads/pL  and  blocked  overnight  with  PBS+  1%  BSA.  25pL  of  blocked  beads  were  pipetted  into  a  96- 
well  plate  with  round  bottomed  wells  and  mixed  with  antigen  diluted  into  block  (or  PBS  for  BoNT  A)  at  a 
concentration  expected  to  be  detected.  The  plate  was  incubated  on  a  plate  shaker  in  the  dark  at 
ambient  temperature  at  800  rpm  for  30  minutes.  Detector  antibody  was  titrated  in  two-fold  increments 
into  block  and  25pL  was  added  to  each  well.  The  plate  was  again  incubated  as  previously  described. 
SAPE  was  titrated  in  two-fold  increments  into  block  and  25pL  was  added  to  each  well.  The  plate  was 
again  incubated  as  previously  described.  The  plate  was  placed  on  a  magnetic  bead  separator  and  the 
beads  were  allowed  to  adhere  for  30  seconds.  With  the  plate  still  on  the  separator,  the  liquid  was 
decanted  and  the  wells  were  washed  twice  with  lOOpL  wash  buffer  (PBS  +  0.1%  Tween-20).  The  plate 
was  removed  from  the  separator  and  the  beads  were  resuspended  in  lOOpL  wash  buffer  by  pipetting  up 
and  down.  75pL  of  each  well  was  analyzed  on  the  MAGPIX  instrument  with  an  additional  internal  wash 
step  with  MAGPIX  Drive  Fluid.  Optimal  biotinylated  antibody  and  SAPE  concentrations  were  chosen  by 
comparing  the  median  fluorescent  intensity  for  each  concentration  of  detector  or  SAPE.  The  optimal 
concentrations  are  summarized  in  the  table  below. 
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Table  41.  Optimal  Concentrations  of  the  biotinylated  detector  antibody  and  conjugate 


Assay 

Biotinylated  Detector  Antibody 
(Hg/mL) 

SAPE  conjugate 
(Hg/mL) 

B.  anthracis 

8 

16 

Y.  pestis 

8 

16 

VAC 

4 

16 

VEE 

2 

16 

BoNT  A 

4 

8 

Step  5:  Comparison  of  washed  to  no-wash  assay 

The  recommended  protocol  for  assays  developed  for  the  MAGPIX  includes  a  microsphere  trapping  and 
wash  step  following  each  incubation.  The  impact  of  removing  the  intermediate  wash  steps  was 
determined  to  be  minimal  using  the  Y.  pestis  assay  as  a  benchmark.  Washing  the  plate  did  result  in 
slightly  increased  median  intensities  and  reduced  the  "hook  effect"  observed  with  high  concentrations 
of  antigen.  The  slight  increase  in  median  intensity  could  result  in  a  lower  LOD  by  increasing  the  intensity 
of  marginally  negative  samples  above  the  three-times  background  threshold,  but  the  background  of  the 
washed  assay  was  higher  than  the  no-wash  assay.  It  was  determined  that  the  increase  in  assay  time  and 
effort  to  perform  the  washing  steps  was  not  worth  the  minimal  increases  in  assay  performance. 

Step  6:  Preliminary  LOD 

A  preliminary  LOD  of  each  assay  was  determined  by  titrating  the  antigen  in  10-fold  increments  and 
testing  each  dilution  in  triplicate  with  the  detector  and  SAPE  at  concentrations  previously  deemed 
optimal.  The  preliminary  LOD  was  set  as  the  concentration  of  antigen  for  which  each  of  the  triplicate 
samples  were  positive.  A  positive  response  was  recorded  if  the  median  intensity  of  the  sample  was 
greater  than  three  times  the  average  intensity  of  triplicate  no-antigen  samples. 

Assay  Sensitivity 

The  Luminex  MAGPIX  assays  were  among  the  most  sensitive  of  all  antibody-based  systems.  The 
sensitivity  for  singleplex  assays  was  similar  to  that  for  the  MSD  point  of  care  Cartridge  Reader.  The  LOD 
for  the  B.  anthracis  assay  was  found  to  be  1.00x10s  CFU/mL,  while  the  LOD  for  Y.  pestis  was  found  to  be 
1.00x10s  CFU/mL.  The  sensitivities  of  these  "home-made"  MAGPIX  bacteria  assays  is  similar  to  the  high 
end  of  enzyme-linked  immunosorbent  assay  (ELISA)  and  lateral  flow  immunoassays  (LFIA),  with  LODs  for 
bacteria  in  the  104-105  CFU  range.  The  viral  assay  LODs  were  found  to  be  l.OOxlO7  for  VAC  and  1.00x10s 
for  VEE  which  is  in  the  106-108  PFU/mL  range  for  viruses  seen  with  ELISA  and  LFIAs.  The  MAGPIX  BoNT  A 
assay  LOD  was  found  to  be  lpg/mL,  which  is  at  the  low  end  of  most  ELISA  and  LFIA  sensitivities. 

The  multiplex  assays  for  bacterial  and  viral  targets  were  approximately  ten-fold  less  sensitive  than  for 
the  individual  singleplex  assays,  while  the  toxin  assay  did  not  lose  sensitivity.  One  possible  explanation 
for  the  BoNT  A  assay  retaining  sensitivity  in  the  multiplex  assays  was  the  increase  in  SAPE  conjugate  to 
16|ig/mL  from  8pg/mL  in  the  singleplex  assay.  The  bacterial  and  viral  multiplex  assays  did  not  have  a 
similar  increase  in  the  SAPE  conjugate  compared  to  the  singleplex  assays,  and  thus,  experienced  lower 
sensitivity.  As  stated  previously,  the  multiplex  assays  were  not  fully  optimized  by  ECBC  staff.  The  results 
of  this  study  may  have  been  more  favorable  if  multiplex  assay  optimization  had  been  performed. 
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by  Meso  Scale  Discovery  (MSD) 


System  Specifications 


Vendor:  Meso  Scale  Discovery  (MSD) 
Website:  www.meso-scale.com 
System  Cost:  Price  on  Request 
Assay  Cost:  Price  on  Request 

Assay  Storage  Requirements:  Refrigeration 

Agents  Tested  per  Assay:  5 

Assay  Shelf  Life:  1  year  at  4  °C 

Sample  Size  Required:  165  |jL 

Type  of  Detection:  Antibody 

Time  to  Result:  30  minutes 

System  Weight:  13  lbs 

Operating  Range:  59-86  °F  (15-30  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  had  successful  operation  in  Mobile  lab  and 
field.  Prototype/Beta  test  unit. 

«  -a- 


System  Description 


The  Cartridge  Reader  is  one  of  the  few  non- 
COTS  systems  included  in  this  testing.  MesoScale 
Diagnostics’  (MSD)  Cartridge  Reader  uses  Multi-Array 
electrochemiluminescence  (ECL)  technology  to  provide 
highly-sensitive  multiplexed  detection  in  a  small  volume  of 
liquid  sample.  The  instrument  carries  out  measurements 
using  single-use  injection-molded  fluidic  cartridges  that  can 
conduct  multiplexed  measurements  of  up  to  12  targets 
in  a  sample.  Each  cartridge  has  integrated  positive  and 
negative  controls.  Integrated  fluidics  on-board  the  cartridge 
allow  for  fully  automated  sample  processing  and  analysis. 


Test  Bed  Review 


As  a  top  performer  and  end  user  favorite,  the  Cartridge 
Reader  was  easy  to  use,  featured  multiplex  capability  and 
consistently  performed  throughout  the  entire  testing. 

Achieving  detection  levels  down  to  lxlO5  cfu/ml  in  both  its 
Bacillus  anthracis  and  Yersinia  pestis  singleplex  assays, 
the  Cartridge  Readers’  sensitivity  was  comparable  to  other 
fully  developed  antibody-based  system.  In  addition  to 
performing  well  in  their  singleplex  assay  design,  MSD  also 
scored  well  in  our  “Multiplex”  category.  This  technology  was 
able  to  detect  all  five  targets  with  no  loss  of  signal  when 
combined  in  a  duplex  format.  During  the  initial  laboratory 
testing,  the  user  interface  displayed  raw  data  and  required 
exporting  the  data  to  an  EXCEL  macro  file  to  calculate  the 
final  results. 

Prior  to  field  testing,  MSD  was  contacted  and  asked  to 
modify  the  user  interface  so  that  results  were  easily 
interpretable  by  the  end  user.  MSD  was  able  to  perform  a 
firmware  upgrade  and  testing  within  one  week.  In  the  field, 
the  end-users  were  impressed  with  the  ease  of  set-up  and 
minimum  amount  of  training  required.  Soldiers  referred  to 
the  system  as  “Superior”  and  a  “Favorite”  of  the  antibody 
based  systems.  One  end-user  commented  that  the  system 
seemed  “almost  too  easy”.  Where  this  system  lacks  the 
sensitivity  of  a  PCR-based  technology  it  excels  in  its  ability 
to  analyze  up  to  12  different  agents  in  one  sample. 

Admittedly  the  current  configuration  of  this  system  was 
not  intended  for  outdoor  use  because  it  offers  no  battery 
option  and  the  system  has  not  been  ruggedized.  However 
these  characteristics  were  minor  concerns  for  the  end- 
users  who  viewed  this  system  as  one  of  their  top  picks. 
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Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

l.OOxlO5 

© 

1.00xl05 

Yersinia  pestis 

l.OOxlO7 

© 

1.00x10s 

Vaccinia 

1.00x10s 

© 

l.OOxlO7 

Venezuelan  Equine 
Encephalitis 

1.00x10s 

© 

1.00x10s 

Clostridium 

Botulinum  Toxin 

7.00xl0'2 

O 

l.OOxlO2 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

l.OOxlO5 

© 

l.OOxlO5 

Yersinia  pestis 

l.OOxlO5 

© 

l.OOxlO5 

Vaccinia 

l.OOxlO7 

© 

>1.00xl08 

Venezuelan  Equine 
Encephalitis 

l.OOxlO8 

© 

1.00x10s 

Clostridium 

Botulinum  Toxin 

l.OOxlO2 

© 

l.OOxlO3 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 

100 


cn 

Ln 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 


MAN  PORTABLE 


MOBILE 

,  L 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  o 


VERY  LIMITED/ 
DIFFICULT 
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Meso  Scale  Discovery 

Overview 

The  Cartridge  Reader  is  one  of  the  few  non-COTS  systems 
included  in  this  testing.  MSD  Cartridge  Reader  uses  Multi- 
Array0  electrochemiluminescence  technology  to  provide 
highly-sensitive  multiplexed  detection  in  a  small  volume  of 
liquid  sample.  The  instrument  carries  out  measurements 
using  single-use  injection-molded  fluidic  cartridges  that  can 
conduct  multiplexed  measurements  of  up  to  12  targets  in  a 
sample.  Each  cartridge  has  integrated  positive  and  negative 
controls.  Integrated  fluidics  on-board  the  cartridge  allow  for 
fully  automated  sample  processing  and  analysis. 

The  Cartridge  Reader  was  tested  to  evaluate  the  technology 
and  verify  vendor  claims  of  LOD  for  five  biothreat  agents  (two  bacterial,  two  viral,  and  one  protein 
toxin).  The  Cartridge  Reader  is  a  simple  to  use  device  that  processes  cassettes  containing  assays  for  each 
target  using  microfluidics.  A  user  pipets  165pL  of  sample  into  the  sample  port,  caps  the  port,  and  inserts 
the  cassette  into  the  device.  Results  are  available  in  approximately  30  minutes  and  are  displayed  on  the 
device  screen  as  positive  or  negative.  The  customized  cartridges  contained  assays  for  all  five  targets 
studied  in  this  evaluation  split  between  two  channels  within  the  cassette.  Each  channel  contains  a 
negative  and  a  positive  control  for  quality  control.  There  is  space  for  a  total  of  six  assays  plus  a  positive 
and  negative  control  on  each  channel;  therefore,  a  cassette  could  be  developed  to  test  for  12  targets 
simultaneously. 

The  LODs  reported  for  each  target  were  taken  from  the  results  of  Joint  Biological  Tactical  Detection 
System  (JBTDS)  Technical  Readiness  Evaluation  (TRE)  09-1  and  are  the  lowest  concentration  that  yielded 
100%  detection  of  the  agent.  The  LOD  of  the  BoNT  A  assay  is  from  work  funded  by  the  National 
Institutes  of  Health  (NIH)/National  Institute  of  Allergy  and  Infectious  Disease  (NIAID)  and  is  for  agent  in 
buffer. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Gamma-irradiated  B.  anthracis  was  diluted  serially  in  10-fold  increments  into  water  and  tested  based  on 
the  flow  chart  in  Figure  6  beginning  with  the  most  concentrated  sample  and  progressing  toward  the 
most  dilute  sample  until  a  negative  result  was  obtained.  Because  the  l.OOxlO4  CFU/mL  sample  was 
negative,  1.00x10s  CFU/mL  was  tested  in  triplicate.  All  three  samples  were  detected  positive;  therefore, 
the  LOD  of  the  device  was  determined  to  be  1.00x10s  CFU/mL,  which  is  in  agreement  with  the  JBTDS 
TRE  data  supplied  by  the  vendor. 
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Table  42.  Bacillus  anthracis  LOD 


Concentration 
(Colony  Forming 
Units  (CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents  (GE) 

Results 

(Positives/Total  Runs) 

l.OOxlO4 

1.65xl03 

3.93xl03 

0/1 

l.OOxlO5 

1.65xl04 

3.93xl04 

3/3 

l.OOxlO6 

1.65x10s 

3.93xl05 

1/1 

l.OOxlO7 

1.65xl06 

3.93xl06 

1/1 

Vendor  Claimed  LOD:  l.OOxlO5  CFU/mL 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma-irradiated  Y.  pestis  was  diluted  serially  in  10-fold  increments  into  water  and  a  single  sample  at 
each  dilution  was  screened  beginning  with  1.30xl08  CFU/mL  and  progressing  to  1.30xl02  CFU/mL. 
Because  the  1.30xl04  CFU/mL  sample  was  the  first  negative  obtained,  two  additional  tests  at  that 
concentration  and  at  the  three  concentrations  immediately  above  it  were  completed  to  determine  the 
consistency  of  the  results.  The  LOD  of  the  device  was  determined  to  be  1.30xl05  CFU/mL.  This  is 
100-fold  better  than  was  determined  during  the  JBTDS  TRE  study,  but  it  is  more  in  line  with  both  the  B. 
anthracis  results  reported  above  and  the  expectations  of  a  bacterial  capture  immunoassay. 

Table  43.  Yersinia  pestis  LOD _ 


Concentration 

Total  CFU 

Total  GE 

Results 

(CFU/mL) 

(Positives/Total  Runs) 

1.30xl02 

2.15X101 

2.98X101 

i/i 

1.30xl03 

2.15xl02 

2.98xl02 

0/1 

1.30xl04 

2.15xl03 

2.98xl03 

0/3 

1.30x10s 

2.15xl04 

2.98xl04 

3/3 

1.30xl06 

2.15x10s 

2.98x10s 

3/3 

1.30xl07 

2.15xl06 

2.98x10s 

3/3 

1.30xl08 

2.15xl07 

2.98xl07 

1/1 

Vendor  Claimed  LOD:  l.OOxlO7  CFU/mL 
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Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  viral  particles  were  diluted  serially  in  10-fold  increments  into  water  and  tested 
based  on  the  flow  chart  in  Figure  6  beginning  with  the  most  concentrated  sample  and  progressing  to  the 
most  dilute  sample  until  a  negative  result  was  obtained.  Because  the  l.OOxlO6  CFU/mL  sample  was 
negative,  l.OOxlO7  CFU/mL  was  tested  in  triplicate.  All  three  samples  were  detected  positive;  therefore, 
the  LOD  of  the  device  was  determined  to  be  l.OOxlO7  CFU/mL.  This  LOD  is  100-fold  higher  than  the  LOD 
determined  during  the  JBTDS  TRE. 

Table  44.  Vaccinia  LOD 


Concentration 
(Plaque  Forming 
Units  (PFU)/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

l.OOxlO6 

1.65x10s 

2.38x10s 

0/1 

l.OOxlO7 

1.65xl06 

2.38xl06 

3/3 

l.OOxlO8 

1.65xl07 

2.38xl07 

1/1 

Vendor  Claimed  LOD:  l.OOxlO5  PFU/mL 


Venezuelan  Equine  Encephalitis 

+  sense  ssRNA,  Alphavirus 

Gamma-irradiated  VEE  viral  particles  were  diluted  serially  in  10-fold  increments  into  water  and  tested 
based  on  the  flow  chart  in  Figure  6,  beginning  with  the  most  concentrated  sample  and  progressing  to 
the  most  dilute  sample  until  a  negative  result  was  obtained.  The  first  test,  performed  using  l.OOxlO8 
PFU/mL,  was  expected  to  be  positive  based  on  results  from  VAC  testing.  The  first  replicate  was  positive, 
so  testing  was  continued  at  l.OOxlO7  PFU/mL.  This  sample  was  negative.  Two  additional  tests  at 
l.OOxlO8  PFU/mL  were  then  performed.  All  three  l.OOxlO8  PFU/mL  samples  were  detected  positive; 
therefore,  the  LOD  of  the  device  was  determined  to  be  l.OOxlO8  CFU/mL,  which  is  in  agreement  with  the 
JBTDS  TRE  data. 

Table  45.  Venezuelan  Equine  Encephalitis  LOD 


Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

l.OOxlO7 

1.65x10s 

1.65x10s 

0/1 

1.00x10s 

1.65xl07 

1.65xl07 

3/3 

Vendor  Claimed  LOD  1.00x10  PFU/mL 


Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

Active  BoNT  A  was  diluted  serially  in  10-fold  increments  into  PBS  and  tested  beginning  at  the  reported 
LOD  and  progressing  as  described  in  Figure  6.  After  getting  negative  results  for  all  three  dilutions  of 
BoNT  A,  fresh  dilutions  were  prepared  the  following  day.  A  lOOng/mL  sample  had  previously  been 
shown  to  be  positive  on  the  system,  so  instead  of  testing  70pg/mL,  additional  lOOng/mL  samples  were 
tested.  All  three  1.00xl02ng/mL  samples  were  detected  positive;  therefore,  the  LOD  of  the  device  was 
determined  to  be  1.00xl02ng/mL,  nearly  1,500-fold  higher  than  had  been  reported  by  MSD.  Of  note 
during  this  assessment,  the  70pg/mL  sample  was  reported,  falsely,  positive  for  VAC. 
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Table  46.  Clostridium  botulinum  toxin  Limit  of  Detection  LOD 


Concentration 
(nanograms  (ng)/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Run) 

7.00x10  1 

1.16x10  1 

0/1 

7.00x10° 

1.16x10° 

0/1 

l.OOxlO2 

1.65X101 

3/3 

Vendor  Claimed  LOD:  70pg/mL 


Multiplex  Detection 

Samples  containing  all  four  inactivated,  gamma  irradiated  agents  were  made  and  tested  in  triplicate  on 
the  custom  assay  designed  by  MSD.  All  four  agents  were  detected  in  all  three  samples  at  the  same 
concentrations  that  had  been  detected  in  the  singleplex  assessment.  This  assessment  was  repeated  with 
samples  containing  BoNT  A  as  described  below. 

Samples  containing  all  agents  diluted  to  the  concentration  found  to  be  the  LOD  during  singleplex  testing 
were  made  and  tested  in  triplicate  on  the  custom  assay  designed  by  MSD.  The  sample  contained 
l.OOxlO5  CFU/mL  B.  anthracis,  l.OOxlO5  CFU/mL  Y.  pestis,  l.OOxlO7  PFU/mL  VAC,  l.OOxlO8  PFU/mL  VEE, 
and  1.00xl02ng/mL  BoNT  A.  B.  anthracis,  Y.  pestis,  and  VEE  were  detected  in  all  three  samples,  whereas 
VAC  was  detected  in  only  one  and  BoNT  A  was  detected  in  none.  Fresh  triplicate  samples  containing 
agents  at  the  same  concentration  as  above,  except  VAC  which  was  at  l.OOxlO8  PFU/mL  and  BoNT  A 
which  was  at  1.00xl03ng/mL.  were  made  and  again  tested  on  the  custom  designed  MSD  assay  cartridge. 
All  agents  were  detected  in  all  three  samples  with  the  modified  concentrations.  For  multiplex  detection, 
it  appears  there  is  interference  between  the  BoNT  A  and  VAC  detection  assays. 


Table  47.  Multiplex  Evaluation  1  -  Without  C.  botulinum  Type  A  toxin 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

B.  anthracis 

1.00x10s 

1.65xl04 

3.93xl04 

3/3 

Y.  pestis 

l.OOxlO5 

1.65xl04 

2.29xl04 

3/3 

Agent 

Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

VAC 

l.OOxlO7 

1.65xl06 

2.38xl06 

3/3 

VEE 

l.OOxlO8 

1.65xl07 

1.65xl07 

3/3 

Table  48.  Multiplex  Evaluation  2  -  With  C.  botulinum  Type  A  toxin 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

B.  anthracis 

1.00x10s 

1.65xl04 

3.93xl04 

6/6 

Y.  pestis 

l.OOxlO5 

1.65xl04 

2.29xl04 

6/6 

Agent 

Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

VAC 

l.OOxlO7 

1.65x10s 

2.38x10° 

1/3 

l.OOxlO8 

1.65xl07 

2.38xl07 

3/3 

VEE 

l.OOxlO8 

1.65xl07 

1.65xl07 

3/3 

Agent 

Concentration 

(ng/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

BoNT  A 

l.OOxlO2 

N/D 

N/D 

0/3 

l.OOxlO3 

N/D 

N/D 

3/3 
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Discussion 
Call  Assignments 

Individual  call  assignments  are  made  by  algorithm  analysis  of  light  emitted  from  the 
electrochemiluminescence  signal.  The  assay  format  in  the  cartridge  is  two  parallel  chambers  with  eight- 
plex  arrays  of  individual  sandwich  immunoassays,  electrical  stimulation  of  a  coordinated  Ru2+  detection 
reagent,  and  capture  of  emitted  light  at  620  nm  by  CCD  camera.  The  arrays  are  configurable,  as 
demonstrated  by  the  ability  of  the  manufacturer  to  rapidly  develop  custom  content  from  CRP's  supply 
of  specific  antibody  reagents. 

Assay  Sensitivity 

The  electrochemiluminescence  assay  technology  employed  with  the  Cartridge  Reader  offers  two 
advantages  over  traditional  fluorescence-based  sandwich  immunoassay.  The  excitation  of  signal  is 
electrical  rather  than  light,  allowing  for  multiple  excitations  per  reporter  molecule  without  the  worry  of 
photobleaching  and  reduced  signal  over  time.  There  is  also  no  signal  loss  or  ambiguity  due  to  spectral 
overlaps  and  related  color-deconvoluting  signal  processing.  Both  of  these  characteristics  would  be 
expected  to  enhance  signal  to  noise  ratios,  resulting  in  more  sensitive  and  reliable  detection. 

However,  the  sandwich  immunoassay  format  is  still  subject  to  limitations  inherent  in  antibody-based 
technologies  such  as  non-specific  binding,  shared  antigen  reactivity  in  near-neighbor  species,  and  cross¬ 
reactivity  in  conserved  non-linear  epitope  motifs.  All  of  these  issues  can  be  minimized  by  blocking, 
affinity  purification  (positive  and  negative  selection),  and  targeted  monoclonal  antibody  production. 
Each  of  these  modifications  has  trade-offs  in  terms  of  sensitivity,  specificity,  cost,  and  development 
time,  which  will  affect  system  performance. 

The  sensitivity  of  the  system  is  in  line  with  traditional  ELISA  and  LFIA  performance,  with  LODs  for 
bacteria  in  the  104-105  CFU  range,  106-107  PFU  for  viruses,  and  >lug/mL  for  protein  toxins.  There  was  a 
false-positive  result  for  VAC  in  the  singleplex  BoNT  A  testing,  which  is  similar  to  a  noted  specificity  issue 
seen  during  the  JBTDS  TRE  09-1.  An  unexpected  loss  in  sensitivity  at  higher  levels  of  multiplexing  was 
noted  for  both  VAC  and  BoNT  A.  In  theory,  multiplex  immunoassays  should  not  have  the  sensitivity  loss 
often  seen  with  multiplex  PCR  systems  caused  by  the  difficulty  in  balancing  multiple  simultaneous 
enzymatic  processes.  Possible  sources  of  this  loss  of  sensitivity  could  be  competition  for  reporter 
reagents,  design  issues  with  the  cartridge  resulting  in  lower  signal  to  noise  ratios,  or  interference  seen 
during  this  testing  between  BoNT  A-  and  VAC-specific  antigen-antibody  pairings. 
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RAPTOR™  Plus 


by  Research  International,  Inc. 


System  Specifications 


Vendor:  Research  International,  Inc. 
Website:  www.researchintl.com 

System  Cost:  $49,500.00 
Assay  Cost:  $150 

Assay  Storage  Requirements:  Refrigeration 
Agents  Tested  per  Assay:  4 
Assay  Shelf  Life:  1  year  at  4  °C 
Sample  Size  Required:  1-2  ml_ 

Type  of  Detection:  Antibody 
Time  to  Result:  28  minutes 
System  Weight:  14.6  lbs  (with  battery) 

Operating  Range:  34  -95  °  F  (1  -  35  °  C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  completed  but  had  recurring  technical  issues. 

Cl  ,?  „  fl  -^^-7  g  q  V 


System  Description 


The  RAPTOR  by  Research  International  is  a  field-ready 
ruggedized  fluorometric  assay  system  about  the  size  and 
weight  of  a  car  battery  that  can  be  used  to  detect  biological 
agents,  chemical  contaminants  or  explosives.  For  biological 
agent  identification,  the  RAPTOR  uses  a  four  channel  wave¬ 
guide  system  with  specific  capture  antibodies  bound  to  an 
immunoassay  “coupon”  for  detection  and  identification  of 
potential  threat  agents.  The  operator  must  prepare  and 
emplace  the  detector  antibody  tubes,  a  pouch  with  running 
buffer  and  a  waste  pouch  prior  to  operation.  The  RAPTOR 
has  two  internal  peristaltic  pumps  that  control  the  fluids’ 
movements  and  the  assay  progression.  The  total  time  for 
set-up,  system  test  and  establishment  of  a  pre-operational 
baseline  reading  is  approximately  30  minutes,  while  the 
sample  run  time  is  14  minutes.  The  instrument  is  not 
quantitative;  however,  detection  signals  are  displayed  as 
negative ,  suspect ,  positive,  and  highly  positive  results, 
providing  some  indication  of  relative  quantity  of  a  particular 
target. 


Test  Bed  Review 


The  laboratory  assessment  for  the  RAPTOR  utilized  its 
Bioassay  Coupon  kit,  which  included  assays  for  Bacillus 
anthracis,  Yersinia  pestis,  Vaccinia  virus  and  BoNT  A. 

The  technical  staff  at  Research  International  was  not 
able  to  develop  a  working  assay  for  Venezuelan  Equine 
Encephalitis  virus.  The  RAPTOR  did  not  fare  well  in  the 
laboratory  assessment  as  positive  results  were  not 
obtained  at  all  for  Bacillus  anthracis  or  Vaccinia  virus  at 
up  to  lOOx  the  claimed  LOD,  while  Yersinia  pestis  was 
only  positive  at  5  x  107  CFU/mL,  or  lOx  the  claimed  LOD. 
The  RAPTOR  required  some  practice  to  become  adept  at 
making  the  proper  fluidics  connections  prior  to  operation. 
The  tubing  was  prone  to  kink  and  color  coded  connectors 
were  mismatched.  Also,  an  interior  module  that  holds  the 
detector  antibody  tubes  had  to  be  frozen  prior  to  operation, 
possibly  causing  inconsistent  assay  conditions  as  the 
instrument  warmed  throughout  the  day. 

The  RAPTOR  was  assessed  in  the  Mobile  Laboratory  and  by 
both  field  operator  groups.  However,  5  of  the  8  operators 
were  unable  to  complete  sample  analysis  because  the 
RAPTOR  failed  the  fluidics  pre-operational  testing  and 
would  not  operate  correctly,  or  failed  during  the  sample 
analysis.  Consequently,  the  operators’  rating  of  “Ease  of 
Use”  varied.  The  manipulation  of  tubing  and  pouches  prior 
to  sample  analysis  was  difficult  for  operators  in  multiple 
glove  layers.  One  operator  reflected,  “You  shouldn’t  have 
to  be  a  mechanic  to  set-up  the  device.”  The  RAPTOR 
received  consistently  positive  ratings  for  data  viewing  and 
interpretation  because  the  surface  display  is  simple  to  read 
and  understand. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

5.ooxio4- 

s.ooxio5 

O 

>1.00xl07 

Yersinia  pestis 

No  Claim 

O 

5.00xl07 

Vaccinia 

l.OOxlO5 

O 

>1.00xl07 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

1-10 

O 

l.OOxlO4 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

>1.00xl07 

O 

>5.00xl06 

Yersinia  pestis 

5.00xl07 

© 

5.00xl07 

Vaccinia 

>1.00xl07 

O 

>1.00xl07 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

©  Validated  Q  Not  Validated 

(<1  log  difference) 


0  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


100 


U1 

Ln 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 


The  RAPTOR  by  Research  International  is  a  field- 
ready  ruggedized  fluorometric  assay  system  about 
the  size  and  weight  of  a  car  battery  that  can  be 
used  to  detect  biological  agents,  chemical 
contaminants,  or  explosives.  For  biological  agent 
identification,  the  RAPTOR  uses  a  four  channel 
wave-guide  system  with  specific  capture 
antibodies  bound  to  an  immunoassay  "coupon" 
for  detection  and  identification  of  potential  threat 
agents.  The  operator  must  prepare  and  emplace 
the  detector  antibody  tubes,  a  pouch  with  running 
buffer  and  a  waste  pouch  prior  to  operation.  The 
RAPTOR  has  two  internal  peristaltic  pumps  that 
control  the  fluids'  movements  and  the  assay 
progression.  The  total  time  for  set-up,  system  test,  and  establishment  of  a  pre-operational  baseline 
reading  is  approximately  30  minutes,  while  the  sample  run  time  is  14  minutes.  The  instrument  is  not 
quantitative;  however,  detection  signals  are  displayed  as  negative,  suspect,  positive,  and  highly  positive 
results,  providing  some  indication  of  relative  quantity  of  a  particular  target. 


The  Research  International  RAPTOR  four  channel  wave-guide  immunoassay  system  running  the  Bioassay 
Coupon  kit,  which  includes  assays  for  Bacillus  anthracis,  Yersinia  pestis,  VAC  virus,  and  BoNT  A,  was 
tested  to  evaluate  the  technology  and  verify  vendor  claims  of  LOD  for  four  biothreat  agents  (two 
bacterial,  one  toxin,  and  one  viral).  The  RAPTOR  was  designed  to  allow  users  to  set-up  the  device,  move 
it  into  a  field  setting,  and  operate  continuously  for  up  to  24  hours  on  battery  power,  but  can  also  be 
used  in  a  laboratory  setting  with  an  A/C  adaptor.  A  single  Bioassay  coupon  kit  can  process  up  to  30 
samples  if  all  results  are  negative.  A  new  kit  must  be  installed  after  each  suspicious  or  positive  result. 
The  coupon  contains  four  wave-guides,  each  functionized  with  a  capture  antibody.  The  device  has  a 
fluidics  compartment  into  which  the  user  loads  rehydrated  fluorophore-labeled  detector  antibodies,  a 
bag  containing  running  buffer,  and  a  waste  bag.  Fluidics  are  manipulated  within  the  device  by  the  use  of 
two  peristaltic  motors.  Complete  set-up,  performance  of  a  system  test  that  checks  for  proper  function  of 
optical  and  fluidic  components,  and  obtaining  a  baseline,  takes  approximately  30  minutes.  A  sample  is 
processed  in  14  minutes,  assuming  the  baseline  is  already  established  during  instrument  set-up.  The 
system  requires  large  inoculums  of  sample  of  l-2mL  which  are  added  directly  to  a  stainless  steel  sample 
inlet. 


Decontamination  of  the  RAPTOR  after  a  positive  detection  of  an  agent  may  prove  problematic.  The 
stainless  steel  sample  port  could  easily  be  damaged  by  bleach  solutions.  Additionally,  the  reagents  are 
recycled  during  sample  analysis  so  that  another  sample  can  quickly  be  analyzed  if  the  initial  sample  is 
negative.  Therefore,  if  an  agent  is  present  but  below  the  LOD,  all  four  detector  antibodies  would  be 
contaminated.  Finally,  the  instructions  indicate  that  in  the  event  of  a  positive  test,  the  assay  coupon 
should  be  removed  and  replaced  by  a  "cleaning  coupon."  Without  first  decontaminating  the  fluidic 
components  of  the  device  with  the  analysis  coupon  in  place,  the  cleaning  coupon  would  also  become 
contaminated  with  agent. 
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Another  drawback  of  the  device  was  the  lack  of  any  internal  controls  or  standards  that  are  run  with  the 
samples.  Without  an  internal  control,  there  was  no  way  to  distinguish  a  true  negative  result  from  a  false 
negative  caused  by  the  presence  of  an  inhibitory  sample  contaminant  or  to  distinguish  a  true  positive 
result  from  a  false  positive  result  caused  by  the  presence  of  a  contaminant  with  intrinsic  fluorescent 
properties. 


The  individual  detector  antibodies  included  in  the  kit  are  in  tubes  capped  with  colored  lids,  but  the 
colors  of  the  kit  components  do  not  match  the  colors  of  the  reagent  tubing  assemblies.  This  led  to 
confusion  when  setting  up  the  instrument.  Additionally,  two  field  users  had  issues  with  kinked  tubing 
when  the  fluidic  compartment  lid  was  put  in  place.  All  users  of  the  instrument  had  issues  with  bubbles 
detected  by  the  system  at  random  times— most  often  while  completing  a  system  test--which  led  to  the 
device  aborting  system  test,  baseline,  and  sample  runs. 

Unfortunately,  the  system  performance,  lack  of  internal  controls,  and  utility  issues,  such  as  tubing  kinks 
and  color  mismatches,  render  this  instrument  all  but  unusable  for  the  intended  purpose. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Gamma-irradiated  B.  anthracis  spores  were  serially  diluted  into  water  (as  indicated  by  manufacturer 
data)  in  10-fold  increments  and  tested  using  the  RAPTOR  Bioassay  Coupon  kit.  The  testing  summarized 
in  the  table  below  represents  several  days  of  testing  because  of  the  number  of  errors  generated  by  the 
device  during  testing.  During  an  initial  assessment  of  the  device,  samples  at  l.OOxlO6  and  l.OOxlO7 
CFU/mL  were  evaluated.  The  l.OOxlO6  CFU/mL  sample  was  negative  while  the  l.OOxlO7  CFU/mL  sample 
was  high  positive  (HI+).  The  assessment  of  the  vendor  claim  of  LOD  was  then  performed.  The  first 
5.00xl04  sample  was  assayed  but  the  device  failed  because  of  bubbles  just  prior  to  the  end  of  the  run. 
The  power  for  the  device  was  cycled  and  the  system  reset.  The  second  5.00xl04  CFU/mL  sample  was 
then  applied  to  the  same  coupon.  The  positive  result  obtained  may  be  due  to  the  initial  sample 
remaining  on  the  chip  and  combining  with  the  agent  in  the  second  sample.  Testing  was  resumed  after  a 
delay  trying  to  determine  the  cause  of  the  bubble  errors  and  began  with  a  5.00xl04  CFU/mL  and 
progressed  to  higher  concentrations  as  described  in  Figure  6.  There  was  not  enough  stock  antigen 
available  to  test  5.00xl07  CFU/mL  in  triplicate— it  was  expected  to  be  positive  based  on  earlier  results 
with  l.OOxlO6  CFU/mL.  Subsequent  testing  with  samples  diluted  to  l.OOxlO6  and  l.OOxlO7  CFU/mL  was 
performed,  each  in  duplicate  because  they  had  been  tested  previously.  B.  anthracis  was  not  detected  in 
any  of  the  four  samples.  The  LOD  of  the  B.  anthracis  assay  was  determined  to  be  greater  than  l.OOxlO7 
CFU/mL  which  is  much  higher,  20-200-fold,  that  the  vendor  claimed  LOD. 
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Table  49.  Bacillus  anthracis  LOD 


Concentration 
(Colony  Forming 
Units  (CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents  (GE) 

Results 

(Positives/Total  Runs) 

5.00xl04 

5.00xl04 

1.19x10s 

1/3  (Pos) 

5.00x10s 

5.00x10s 

1.19x10s 

0/1 

1.00x10s 

1.00x10s 

2.38x10s 

0/3 

5.00x10s 

5.00x10s 

1.19xl07 

0/1 

l.OOxlO7 

l.OOxlO7 

2.38xl07 

1/3  (HI+) 

Vendor  Claimed  LOD:  5.00xl04  -  5.00x10s  CFU/mL 

Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma-irradiated  Y.  pestis  cells  were  serially  diluted  into  water  (as  indicated  by  manufacturer  data)  in 
10-fold  increments  and  tested  using  the  RAPTOR  Bioassay  Coupon  kit  following  the  testing  flow  chart, 
Figure  6.  Assessment  of  the  LOD  was  performed  starting  at  5.00xl06  CFU/mL  because  of  the  apparent 
low  sensitivity,  as  demonstrated  by  the  false  negative  results  of  B.  anthracis  testing.  While  the  5.00xl06 
CFU/mL  sample  was  negative,  the  5.00xl07  CFU/mL  sample  was  reported  as  HI+  for  all  three  tests.  It 
appears  the  device  has  a  small  dynamic  range  because  the  results  are  reported,  in  increasing 
concentration  of  agent,  Negative,  Suspicious,  Positive,  and  HI+.  The  LOD  of  the  assay  was  reported  in 
ng/mL  FI  antigen  which  cannot  be  extrapolated  to  CFU/mL.  It  is  unknown  whether  the  level  of  detection 
determined  in  this  assessment  is  equivalent  to  the  LOD  reported  by  the  manufacturer. 


Table  50.  Yersinia  pestis  LOD 


Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

5.00x10s 

5.00x10s 

6.93x10s 

0/1 

5.00xl07 

5.00xl07 

6.93xl07 

3/3  (HI+) 

Vendor  Claimed  LOD:  l-5ng/mL  FI  antigen 


Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  viral  particles  were  serially  diluted  into  assay  buffer  (8.3mM  Phosphate  buffer, 
pH  7.2,  0.05%  (w/v)  Triton  X-100)  in  10-fold  increments  and  tested  using  the  RAPTOR  Bioassay  Coupon 
kit  as  described  in  Figure  6  starting  at  the  vendor  claimed  LOD.  VAC  was  not  detected  in  any  of  the 
samples  tested.  An  additional  l.OOxlO7  PFU/mL  sample  was  analyzed  on  the  device  using  the  High 
Sensitivity  Assay.  Again,  VAC  was  not  detected.  Testing  was  not  completed  at  concentration  higher  than 
l.OOxlO7  PFU/mL  because  of  the  large  sample  volume  required  and  moderate  concentration  of  VAC 
stock  antigen  available.  The  vendor  claimed  LOD  could  not  be  confirmed. 
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Table  51.  Vaccinia  LOD 


Concentration 
(Plaque  Forming 
Units  (PFU)/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

l.OOxlO5 

1.00x10s 

1.44xl05 

0/1 

l.OOxlO6 

1.00x10s 

1.44xl06 

0/1 

l.OOxlO7 

l.OOxlO7 

1.44xl07 

0/1 

Vendor  Claimed  LOD:  1.00x10s  PFU/mL 


Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

No  Assay  Available 

Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

Active  BoNT  A  was  diluted  serially  in  10-fold  increments  into  PBS  and  tested  beginning  at  10-100  times 
the  published  LOD  because  the  device  had  sub-optimal  sensitivity  when  detecting  bacterial  and  viral 
antigens  previously.  Testing  progressed  as  described  in  Figure  6.  BoNT  A  was  detected  in  the 
10,000ng/mL  sample,  but  not  in  the  more  dilute  samples.  The  1.00xl04ng/mL  sample  was  not  tested  in 
triplicate  because  of  the  large  sample  volume  required  and  difficulties  decontaminating  the  instrument 
due  to  its  microfluidic  design.  The  LOD  of  the  device  was  determined  to  be  no  lower  than 
1.00xl04ng/mL,  nearly  1,000  to  10,000-fold  higher  than  the  vendor  claimed  LOD. 


Table  52.  Clostridium  botulinum  toxin  Limit  of  Detection  LOD 


Concentration  (nanograms 
(ng)/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

l.OOxlO2 

l.OOxlO2 

0/1 

l.OOxlO3 

l.OOxlO3 

0/1 

l.OOxlO4 

l.OOxlO4 

1/1 

Vendor  Claimed  LOD:  l-10ng/mL 


Multiplex  Detection 

Multiplex  testing  was  completed  with  samples  containing  gamma-irradiated  B.  anthracis  and  Y.  pestis 
because  of  the  large  amount  of  BoNT  A  needed  to  include  the  target  in  the  assessment.  VAC  was  also 
excluded  because  it  was  not  detected  in  any  sample  assessed.  Although  B.  anthracis  was  not  expected 
to  be  detected  at  the  concentration  used  for  the  multiplex  assessment,  B.  anthracis  was  included  to 
determine  whether  the  presence  of  additional  targets  interfered  with  the  detection  of  Y.  pestis.  The 
assays  were  only  performed  in  duplicate  because  of  difficulties  with  bubbles  within  the  microfluidic  lines 
of  the  device.  It  appears  that  the  presence  of  nonspecific  agent  does  not  interfere  with  the  detection  of 
Y.  pestis,  although  VAC  was  falsely  identified  as  being  present  in  one  of  the  samples. 


Table  53.  Multiplex  Evaluation 


Agent 

Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

B.  anthracis 

5.00xl06 

5.00xl06 

1.19xl07 

0/2 

Y.  pestis 

5.00xl07 

5.00xl07 

6.93xl07 

2/2  (HI+) 
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Discussion 


Call  Assignments 

Individual  call  assignments  for  organisms  on  the  multiplex  tickets  are  made  based  upon  fluorescent 
signal  derived  from  sandwich  immunoassay  with  a  fluorescent  reporter  on  optical  wave-guides. 
Processing  of  the  signal  is  performed  to  derive  negative,  suspect,  positive,  and  highly  positive  results. 
The  lack  of  internal  controls  made  it  difficult  to  evaluate  negative  results  generated  at  10  to  100-fold 
above  stated  LODs.  The  observed  high  rate  of  false-negative  results  is  concerning,  as  they  give  the 
impression  of  the  absence  of  agent  when  it  may  present  at  extremely  high  concentrations.  A  false 
positive  result  in  the  absence  of  target  is  also  of  concern,  as  it  raises  the  possibility  that  high 
concentration  of  non-specific  protein  targets  may  result  in  spurious  positive  results  requiring  a  response. 
The  distinction  between  system  calls  may  be  unreliable  with  limited  dynamic  range,  as  all  true  positive 
results  were  highly  positive  in  single  ten-fold  dilution  above  negative  results.  To  troubleshoot  the 
problems  with  bubbles  in  the  tubing,  data  was  sent  to  Research  International  for  analysis.  Additionally, 
the  device  was  connected  to  a  computer  with  diagnostics  software,  but  no  root  cause  was  found. 

Assay  Sensitivity 

Assay  sensitivity  was  extremely  poor,  limited  to  levels  high  enough  that  bulk  contamination  of  the  target 
of  interest  would  be  required  and  the  resulting  viscosity  of  the  bulk  reference  materials  could  limit  the 
very  upper  limit  of  testing.  Two  organisms  (B.  anthracis  and  VAC)  were  not  detected  at  any 
concentration  during  the  evaluation. 

The  root  cause  of  the  lack  of  sensitivity  is  unknown,  and  investigations  will  be  significantly  inhibited  by 
the  lack  of  internal  controls  to  discriminate  true  negatives  from  assay,  instrument,  reagent,  or 
consumable  failures.  One  possible  source  of  failure  is  the  delicacy  of  wave-guide  systems,  which  have 
been  shown  to  be  subject  to  fouling  in  non-pristine  samples.  Fluidics  issues  and  ambiguity  of  system 
architecture  were  also  found  on  the  instrument. 
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SpinDx™ 


by  Sandia  National  Laboratory 


System  Specifications 


Vendor:  Sandia  National  Laboratory 

Website:  N/A 

System  Cost:  To  be  determined  (non-COTS  device) 

Assay  Cost:  $2  per  disk 

Assay  Storage  Requirements:  4°C 

Agents  Tested  per  Assay:  Up  to  20  possible 

Assay  Shelf  Life:  6  months  at  4  °C 

Sample  Size  Required:  2  pL 

Type  of  Detection:  Fluorescent  labeled  antibodies 

Time  to  Result:  20  minutes 

System  Weight:  3.5  lbs  (system  also  requires  laptop) 
Operating  Range:  59-86  °  F  (15  -  30  °  C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  System  is  early  prototype  unit,  components  are  not 
final.  Assay  development  in  progress. 

Q  £  ifii  7  A 


System  Description 


The  SpinDx  by  Sandia  National  Laboratory  is  a  non- 
COTS  system  designed  primarily  for  rapid  diagnostics 
in  a  clinical  or  point-of-care  setting.  The  system  utilizes 
a  spinning  disk,  or  “lab  on  a  CD”,  to  draw  a  sample 
through  a  separation  matrix  while  simultaneously  binding 
to  fluorescently  labeled  antibody-bead  constructs.  The 
SpinDx  uses  LED  illumination  and  a  photodiode  to  detect 
the  target.  Preliminary  results  from  Sandia  show  the 
system  to  have  greater  detection  sensitivity  than  standard 
ELISAs.  The  SpinDx  has  also  been  used  to  separate  whole 
blood  samples  for  cell  counts  and  other  clinical  analyses. 
Operation  of  the  system  requires  little  or  no  training,  and 
samples  require  no  preparation  except  for  mixing  with 
the  analytical  matrix.  The  SpinDx  is  controlled  by  a  laptop 
computer  via  wireless  (Bluetooth)  communications.  With 
an  analysis  time  of  less  than  20  minutes,  no  sample 
preparation,  battery  power  and  small  size,  the  SpinDx  has 
potential  to  be  a  prototypical  mobile  laboratory  or  even 
hand-held  instrument. 


Test  Bed  Review 


The  results  of  the  laboratory  assessment  indicate  the 
SpinDx  is  still  at  the  developmental  prototype  stage.  Sandia 
utilized  Critical  Reagent  Program  antibodies  to  create 
assays  for  all  5  targets;  however,  the  assays  were  not 
able  to  definitively  detect  any  of  the  targets  in  the  current 
configuration  of  the  SpinDx.  The  analytical  matrix  was 
rather  gel-like,  and  adding  3  uL  of  the  sample-matrix  mix 
to  the  port  on  the  disk  was  difficult  and  not  precise.  Sandia 
supplied  a  specific  pipet  and  plastic  tips  to  load  the  device, 
but  the  sample  loading  step  could  still  be  improved.  The 
operation  of  the  device  was  guided  by  software  on  a  laptop 
computer  connected  wirelessly  to  the  SpinDx.  Because  of 
concerns  about  the  software  measurement  of  analytical 
beads  after  the  “spin”,  Sandia  provided  calibration  beads. 
However,  the  analytical  software  indicated  either  the 
calibration  beads  did  not  run  properly  in  the  disks  or  the 
detection  algorithm  was  errant.  The  battery  re-charger 
connection  interferes  with  closing  of  the  device  lid.  The 
system’s  sample  matrix  loading  and  the  analytical  software 
both  seemed  in  need  of  improvement  for  environmental 
sample  analysis. 

The  SpinDx  was  assessed  by  the  Mobile  laboratory 
operators,  but  not  by  the  field  operators.  Because  of  some 
difficulty  in  mixing  the  sample  with  analytical  matrix  and 
pipeting  into  the  disk  port,  as  well  as  running  the  control 
software,  the  operators  rated  the  “Ease  of  Use”  and  “Data” 
viewing  and  interpreting  attributes  as  being  only  FAIR. 

The  operators  rated  “Training  Simplicity”,  “Safety”  and 
“Maintenance”  all  as  EXCELLENT.  The  operators  also  were 
favorable  to  the  overall  design  of  the  instrument  and  the 
short  musical  tune  that  denotes  completion  of  an  assay. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program. 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

N/A 

O 

N/A 

Yersinia  pestis 

N/A 

O 

N/A 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

N/A 

O 

N/A 

Yersinia  pestis 

N/A 

O 

N/A 

Vaccinia 

N/A 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

N/A 

O 

N/A 

Clostridium 

Botulinum  Toxin 

N/A 

O 

N/A 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 

100 


_OJ 

01 

in 


©  Validated  f%  Not  Validated 

(<1  log  difference) 


Q  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 


development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 

The  SpinDx  by  Sandia  National  Laboratories  is  a  non-COTS 
system  designed  primarily  for  rapid  diagnostics  in  a  clinical  or 
point-of-care  setting.  The  system  utilizes  a  spinning  disk,  or 
"lab  on  a  CD,"  to  draw  a  sample  through  a  separation  matrix 
while  simultaneously  binding  to  fluorescently  labeled 
antibody-bead  constructs.  The  SpinDx  uses  LED  illumination 
and  a  photodiode  to  detect  the  target.  Preliminary  results 
from  Sandia  show  the  system  to  have  greater  detection 
sensitivity  than  standard  ELISAs.  The  SpinDx  has  also  been 
used  to  separate  whole  blood  samples  for  cell  counts  and 
other  clinical  analyses.  Operation  of  the  system  requires  little 
or  no  training  and  samples  require  no  preparation  except  for 
mixing  with  the  analytical  matrix.  The  SpinDx  is  controlled  by  a 
laptop  computer  via  wireless  (Bluetooth)  communications.  With  an  analysis  time  of  less  than  20 
minutes,  no  sample  preparation,  battery  power,  and  small  size,  the  SpinDx  has  potential  to  be  a 
prototypical  mobile  laboratory  or  even  hand-held  instrument. 


The  SpinDX  assay  format  is  a  single  antibody  capture  assay  contained  within  a  spinning  flat  disc 
containing  20  reaction  lanes  laser  etched  and  radiating  from  the  center  of  the  disc.  A  sample  is  applied 
to  a  slot  near  the  center  of  the  disc  and,  as  the  disc  spins,  the  sample  mass  passes  through  a  sieving 
matrix  and  binds  to  specific  antibodies  for  a  target  of  interest.  The  sample-antibody  mixture  passes 
through  the  sieving  matrix  to  the  outward  point  of  the  sample  lane,  where  fluorescence  is  read  by  the 
photodiode.  Each  lane,  in  theory,  could  support  up  to  20  assays  concurrently  based  on  fluorescence 
labeling  of  the  antibodies  and  the  capabilities  of  the  photodiode. 


The  SpinDX  provided  by  Sandia  National  Laboratories  was  a  prototype  device  and  still  under 
development.  While  all  other  instruments  in  this  study  had  been  obtained  by  the  fall  of  2012,  the 
SpinDX  production  was  delayed  and  the  prototype  device  was  not  delivered  until  early  February  2013, 
just  prior  to  the  mobile  laboratory  assessment.  As  such,  the  SpinDX  did  not  conclusively  detect  targets 
among  all  the  samples  that  were  tested.  Future  iterations  of  the  SpinDX  should  correct  the  assay 
performance  problems  of  the  spin  disk  mechanism. 


The  SpinDX  was  assessed  in  the  Mobile  laboratory  scenario  for  usability,  but  not  the  functionality  or 
ability  to  accurately  detect  target  agents. 

Data 

No  usable  data  was  obtained.  During  initial  evaluation  of  Yersinia  pestis  assays,  the  photodiode  signals 
were  not  compatible  with  accurate  data  collection  from  the  sample  disc.  Thus,  the  singleplex  and 
multiplex  assays  were  not  performed. 

Discussion 

Call  Assignments 

The  SpinDX  is  designed  to  make  detection  calls  based  on  an  increase  in  fluorescent  signal  over  baseline 
levels  as  detected  by  the  internal  photodiode.  A  laptop  computer  communicates  with  the  SpinDX  via 
Bluetooth  and  runs  software  that  controls  SpinDX  operations  and  collects  and  interprets  signals  from 


124 

Approved  for  Unlimited  Distribution 
Unclassified 


ANTIBODY  TECHNOLOGIES 


Sandia 

National 

Laboratories 


the  device.  There  is  no  analysis  software  included  on  the  present  prototype  iteration;  therefore, 
detection  algorithms  are  left  to  the  user  to  develop. 


Assay  Sensitivity 

Since  no  assays  were  able  to  be  evaluated,  the  assay  sensitivity  is  unknown  for  this  prototype  version. 
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SPIRIT™ 


by  Seattle  Sensors  Corporation 


System  Specifications 


Vendor:  Seattle  Sensors  Corporation 
Website:  www.seattlesensors.com 

System  Cost:  $35,000.00 
Assay  Cost:  $423.00 

Assay  Storage  Requirements:  Refrigeration 
Agents  Tested  per  Assay:  One 
Assay  Shelf  Life:  6  months  at  4  °C 
Sample  Size  Required:  100-150  p L 
Type  of  Detection:  Antibody 
Time  to  Result:  5-10  minutes 
System  Weight:  3  lbs 
Operating  Range:  59-86  °F  (15-30  °C) 


Technology  Readiness  Level  (TRL) 


The  Technology  Readiness  Level  has  been  determined  by  a 
subject  matter  expert  panel  analyzing  all  relevant  data  and  rated 
according  to  an  interpretation  of  the  Technology  Readiness 
Assessment  (TRA)  Guidance  document  prepared  by  U.S.  DoD’s 
ASD  (R&E).  Software  is  not  yet  final.  System  not  tested  to  full 
capability. 


System  Description 


The  SPIRIT  by  Seattle  Sensors  Systems  Corporation  is  a 
shoe-box  sized  biological  agent  identifier  that  uses  surface 
plasmon  resonance  (SPR)  to  detect  and  measure  binding, 
such  as  between  an  antibody  and  a  specific  bacteria.  The 
SPIRIT  has  condensed  the  research  laboratory  technique 
of  SPR  to  a  portable  device  capable  of  detecting  bacteria, 
viruses  or  toxins  from  complex  samples  by  utilizing  Texas 
Instruments’  Spreeta  SPR  chips.  A  laptop  computer  is  used 
to  control  the  peristaltic  pumps  and  valves  and  to  regulate 
the  flow  of  sample  and  buffer  onto  the  SPR  flowcells  such 
that  each  sample  may  be  analyzed  within  25  minutes. 

The  operator  monitors  the  SPR  signals  through  a  graphic 
display  and  post-run  data  analysis.  The  system  allows  for 
regeneration  of  the  Spreeta  chip  surfaces,  such  that  up 
to  100  samples  may  be  analyzed  before  the  chip  must  be 
replaced.  Among  the  mobile  and  man-portable  detection 
systems,  SPIRIT  has  relatively  fast  assay  times,  a  small 
footprint  and  battery-power. 


Test  Bed  Review 


Seattle  Sensors  Systems  developed  assays  for  all  5  test 
agents  through  binding  antibodies  from  the  Critical  Reagent 
Program  to  individual  SPR  chips.  The  software  version 
included  with  the  SPIRIT  did  not  allow  full  functionality; 
consequently,  only  singleplex  detection  capability  was 
assessed  on  this  version  of  the  SPIRIT.  The  laboratory 
assessment  utilized  individual  targets  on  channel  4  and 
a  calibration  control  on  channel  3,  while  channels  1  and 
2  were  not  configured  for  data  collection  in  this  software 
release.  The  set-up,  priming  and  calibration  of  the  SPIRIT 
took  about  30  minutes,  while  sample  data  collection  took 
approximately  25  minutes.  The  SPIRIT  was  at  the  lower 
end  of  sensitivity  among  the  biological  agent  identifiers 
as  bacteria  samples  were  detected  at  107  CFU/mL  and 
Vaccinia  virus  inconsistently  identified  at  107  PFU/mL. 
Meanwhile,  VEE  was  not  detected  at  108  PFU/mL  or  BoNT  A 
at  1  ug/mL.  The  SPIRIT  required  some  training  and  practice 
for  the  operators  to  become  comfortable  with  performing 
assays. 

The  SPIRIT  received  generally  favorable  usability  scores 
from  Mobile  laboratory  operators,  with  an  EXCELLENT  and 
FAIR  rating  for  “Ease  of  Use”  and  GOOD  or  EXCELLENT 
scores  for  Data,  Training,  Safety  and  Maintenance 
categories.  The  operators  had  some  troubles  with 
the  injection  of  sample  to  the  port  and  experienced 
inconsistent  internal  peristaltic  pump  pressures.  The  SPIRIT 
was  sensitive  to  work  surface  vibrations  perturbing  the 
data  collection.  Of  importance  to  sample  size  requirements, 
one  operator  was  concerned  that  “1  mL  of  sample  was 
needed”  which  may  use  too  much  of  their  collected  sample. 
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Laboratory  Limit  Of  Detection  (LOD) 
Validation 


Singleplex  validation  of  vendor  LOD  claims  and  Multiplex 
replication  of  singleplex  LODs  in  a  laboratory  setting.  Targets 
were  inactivated  pathogens  from  the  U.S.  DoD’s  Critical  Reagent 
Program.  (Units:  Bacteria  =  CFU/mL,  Virus  =  PFU/mL, 
toxin  =  ng/mL) 


SINGLEPLEX 

Agent 

Vendor 

Claimed 

LOD 

Validation 
of  Claimed 
LOD 

Actual  LOD 

Bacillus  anthracis 

No  Claim 

O 

>1.00xl07 

Yersinia  pestis 

No  Claim 

O 

l.OOxlO7 

Vaccinia 

No  Claim 

O 

1.00x10s 

Venezuelan  Equine 
Encephalitis 

No  Claim 

O 

>1.00xl08 

Clostridium 

Botulinum  Toxin 

No  Claim 

O 

>1.00x10® 

MULTIPLEX 

Agent 

Actual 

Singleplex 

LOD 

Actual  LOD 
Achieved 

Multiplex 

LOD 

Bacillus  antracis 

>1.00xl07 

O 

N/A 

Yersinia  pestis 

l.OOxlO7 

O 

N/A 

Vaccinia 

1.00x10s 

O 

N/A 

Venezuelan  Equine 
Encephalitis 

>1.00xl08 

O 

N/A 

Clostridium 

Botulinum  Toxin 

>1.00xl03 

O 

N/A 

Laboratory  Usability  Scores 


Weighted  scoring  is  based  on  laboratory  data  compiled  and 
rated  by  Army  Subject  Matter  Experts. 


U1 

Ln 


©  Validated  Q  Not  Validated 

(<1  log  difference) 


Q  Not  Validated  Q  No  Claim 
(<1  log  difference) 


Man  Portable  and  Mobile  Usability 


Each  technology  has  been  evaluated  for  usability  in  the  field  for 
hand-held/man  portable  and  mobile  laboratory  settings.  The 
ratings  are  based  on  input  from  multiple  Army  field  operators 
and  Subject  Matter  Experts 

MAN  PORTABLE 

v  c  i  7  3 


MOBILE 


Flexibility  to  Add  New  Assays 


Each  technology  has  been  evaluated  for  assay  flexibility  as 
determined  by  multiple  sources  of  assays  and  ease  of  assay 
development.  The  rankings  are  based  on  input  from  multiple 
Army  civilian  users.  2 


VERY  LIMITED/ 
DIFFICULT 


UNLIMITED/ 

SIMPLE 
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Overview 

The  SPIRIT  by  Seattle  Sensors  Systems  Corporation  is  a  shoe- 
box  sized  biological  agent  identifier  that  uses  surface 
plasmon  resonance  (SPR)  to  detect  and  measure  binding, 
such  as  between  an  antibody  and  a  specific  bacteria.  The 
SPIRIT  has  condensed  the  research  laboratory  technique  of 
SPR  to  a  portable  device  capable  of  detecting  bacteria, 
viruses  or  toxins  from  complex  samples  by  utilizing  Texas 
Instruments'  Spreeta  SPR  chips.  A  laptop  computer  is  used 
to  control  the  peristaltic  pumps  and  valves  and  to  regulate 
the  flow  of  sample  and  buffer  onto  the  SPR  flowcells,  such 
that  each  sample  may  be  analyzed  within  25  minutes.  The 
operator  monitors  the  SPR  signals  through  a  graphic  display 
and  post-run  data  analysis.  The  system  allows  for  regeneration  of  the  Spreeta  chip  surfaces  and  up  to 
100  samples  may  be  analyzed  before  the  chip  must  be  replaced.  Among  the  mobile  and  man-portable 
detection  systems,  SPIRIT  has  relatively  fast  assay  times,  a  small  footprint,  and  battery-power. 

The  Seattle  Sensor  Systems  SPIRIT  platform  running  a  customized  SPR  2000  (Spreeta)  Sensor  Chip  with 
existing  antibodies  was  tested  to  evaluate  the  technology  and  verify  vendor  claims  of  LOD  for  five 
biothreat  agents  (two  bacterial,  two  viral,  and  one  protein  toxin).  Four  Spreeta  chips  were  installed  on 
the  SPIRIT  to  complete  the  fluidic  circuit  and  one  of  the  four  was  assigned  as  the  reference  electrode  to 
distinguish  signal  from  nonspecific  background.  Therefore,  the  device  had  the  capability  of  detecting 
three  agents  simultaneously  plus  a  single  reference.  The  analysis  software,  written  specifically  for  this 
assessment,  only  monitored  Spreeta  chips  three  (reference)  and  four  (test).  Because  of  this  limitation, 
no  assessment  of  multiplex  capability  was  performed.  The  controller  software  allowed  the  user  to  select 
one  of  four  pre-programmed  methods  or  control  the  device  manually.  During  operation,  the  user  was 
required  to  install  a  run  buffer,  chip  reconditioning  buffer,  and  empty  waste  bottle  onto  the  system,  all 
of  which  were  included  in  the  "kits,"  and  equilibrate  the  chips  and  tubing.  All  of  these  steps  were 
accomplished  using  manual  controls.  Additionally,  the  antibody  coated  Spreeta  chips  were  rehydrated  in 
run  buffer  for  two  minutes,  installed  into  slot  four,  and  referenced  against  a  high  refractive  index 
solution  (also  included  in  the  kit).  Once  the  chips  were  equilibrated,  the  user  selected  and  started  the 
proper  program  and  injected  the  sample  at  the  appropriate  time.  The  SPIRIT  required  a  minimum  250pL 
sample  to  fill  an  injection  loop  situated  below  the  instrument  panel,  but  the  manufacturer 
recommended  the  use  of  samples  of  approximately  lmL  so  that  the  loop  was  rinsed  and  the  edge  effect 
caused  by  the  mixing  of  buffer  and  sample  were  minimized.  The  programs  utilized  for  this  assessment 
were  capture  assays  without  signal  amplification  and  were  completed  in  approximately  25  minutes. 

In  the  absence  of  an  expert  user,  analysis  of  the  run  data  must  be  completed  utilizing  accessory  software 
to  determine  whether  an  agent  has  been  detected.  The  software  evaluates  the  SPR  signal  during  the 
binding  step  of  the  program  and  outputs  a  slope,  fit,  and  detection  message.  The  increase  in  SPR  signal 
during  binding  is  directly  proportional  to  amount  of  agent  present.  Slope  values  greater  than  0.010  are 
considered  positive  for  agent  while  slope  values  between  0.005  and  0.0099  are  considered  indication  of 
potential  presence  or  weak  detection.  There  are  no  guidelines  for  acceptable  fit  values. 

The  abilities  to  detect  bacterial,  viral,  and  protein  toxins  and  to  provide  an  orthogonal  confirmatory 
technology  to  more  common  nucleic  acid-based  systems  are  both  desirable  characteristics  of  the 
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system.  In  addition,  SPR  is  not  a  commonly  used  detection  scheme,  which  provided  another  technology 
to  evaluate  for  the  program. 

Unfortunately,  the  system  did  not  provide  consistent  and  sensitive  detection  of  bacteria,  viruses,  or 
protein  toxin  at  any  concentration.  The  system  software  was  difficult  to  use  and  poorly  documented. 
Finally,  not  all  of  the  functionality  was  tested,  as  only  two  of  the  four  instrument  channels  were 
available  for  use  because  of  the  analysis  software  on  this  SPIRIT  version.  In  effect,  the  TRL  of  the 
instrument  could  only  be  considered  four  to  five,  well  below  COTS/GOTS  expectations. 

Data 

Bacillus  anthracis 

Gram  positive,  spore  forming  bacilli 

Gamma-irradiated  B.  anthracis  was  diluted  serially  in  10-fold  increments  into  Run  Buffer  (PBS  +  0.1% 
Tween-20)  and  tested  based  on  the  flow  chart  in  Figure  6  beginning  at  l.OOxlO6  CFU/mL.  B.  anthracis 
was  not  detected  in  the  l.OOxlO6  CFU/mL  sample;  therefore,  l.OOxlO7  CFU/mL  sample  was  tested.  B. 
anthracis  was  detected  in  this  sample,  so  the  concentration  was  tested  two  additional  times.  B. 
anthracis  was  detected  in  one  of  the  two  samples,  thus  the  LOD  was  stated  to  be  greater  than  l.OOxlO7 
CFU/mL.  Testing  of  concentrations  above  this  level  was  not  possible  due  to  the  large  sample  volume 
required. 


Table  54.  Bacillus  anthracis  LOD 


Concentration  (Colony 
Forming  Units 
(CFU)/mL) 

Total  CFU 

Total  Genome 
Equivalents  (GE) 

Results 

(Positives/Total  Runs) 

1.00x10s 

2.50x10s 

5.95x10s 

0/1 

l.OOxlO7 

2.50xl06 

5.95xl06 

2/3 

Vendor  Claimed  LOD:  N/A 


Yersinia  pestis 

Gram  negative,  rod-shaped  bacterium 

Gamma-irradiated  Y.  pestis  was  diluted  serially  in  10-fold  increments  into  run  buffer  (PBS  +  0.1%  Tween- 
20)  and  tested  based  on  the  flow  chart  in  Figure  5Figure  6  beginning  at  l.OOxlO6  CFU/mL.  Y.  pestis  was 
not  detected  in  the  l.OOxlO6  CFU/mL  sample;  therefore,  l.OOxlO7  CFU/mL  sample  was  tested.  Y.  pestis 
was  detected  in  all  three  of  the  triplicate  samples,  thus  the  LOD  was  stated  to  be  l.OOxlO7  CFU/mL. 


Table  55.  Yersinia  pestis  LOD 


Concentration 

(CFU/mL) 

Total  CFU 

Total  GE 

Results 

(Positives/Total  Runs) 

1.00x10s 

2.50x10s 

3.46x10s 

0/1 

l.OOxlO7 

2.50x10s 

3.46x10s 

3/3 

Vendor  Stated  LOD:  N/A 
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Vaccinia 

dsDNA  Orthopox  virus,  Smallpox  [Variola]  simulant 

Gamma-irradiated  VAC  was  diluted  serially  in  10-fold  increments  into  run  buffer  (PBS  +  0.1%  Tween-20) 
and  tested  based  on  the  flow  chart  in  Figure  6  beginning  at  l.OOxlO7  PFU/mL  VAC  was  detected  in  the 
first  sample,  though  the  fit  of  the  slope  was  poor,  so  testing  continued  with  the  l.OOxlO6  PFU/mL 
sample.  The  initial  sample  at  l.OOxlO6  PFU/mL  was  also  positive  but  the  slope  appeared  much  larger 
than  expected  (0.072  versus  0.025  for  the  l.OOxlO7  PFU/mL  sample),  there  was  a  large  amount  of  noise 
in  the  signal,  and  the  fit  was  extremely  poor  (0.278).  Therefore,  the  l.OOxlO6  PFU/mL  sample  was 
repeated  rather  than  testing  a  l.OOxlO5  PFU/mL  sample.  VAC  was  not  detected  in  the  repeat.  Additional 
replicates  at  l.OOxlO7  PFU/mL  were  then  tested  and  agent  was  detected  in  neither.  After  the  completion 
of  the  final  l.OOxlO7  PFU/mL  sample  program,  the  system  automatically  restarted  the  program  without 
user  input.  No  additional  sample  was  injected  into  the  sample  loop.  VAC  was  detected  in  this  blank 
sample  and  the  fit  was  the  best  of  all  runs  thus  far  completed.  This  detection  was  considered  a  false 
positive.  A  single  sample  at  l.OOxlO8  PFU/mL  was  then  analyzed.  This  sample  was  weakly  positive  (slope 
0.009)  so  the  LOD  of  the  device  was  considered  to  be  >1.00xl08  PFU/mL.  Testing  of  concentrations 
above  this  level  was  not  possible  due  to  the  large  sample  volume  required. 


Table  56.  Vaccinia  LOD 


Concentration 
(Plaque  Forming  Units 
(PFU)/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

l.OOxlO6 

2.50x10s 

3.61x10s 

1/2 

l.OOxlO7’ 

2.50x10s 

3.61x10s 

2/4 

O*  * 

1.00x10 

2.50xl07 

3.61xl07 

1/1 

*  False  positive  blank  sample  observed 
**  Weak  positive  signal 
Vendor  Stated  LOD:  N/A 


Venezuelan  Equine  Encephalitis 
+  sense  ssRNA,  Alphavirus 

Gamma-irradiated  VEE  viral  particles  were  diluted  into  run  buffer  (PBS  +  0.1%  Tween-20)  and  tested 
starting  at  l.OOxlO8  PFU/mL  because  of  the  poor  results  obtained  with  VAC  viral  particles.  VEE  was  not 
detected  in  the  l.OOxlO8  PFU/mL  sample,  thus  the  LOD  was  stated  to  be  greater  than  l.OOxlO8  PFU/mL. 
Testing  of  concentrations  above  this  level  was  not  possible  due  to  the  large  sample  volume  required. 


Table  57.  Venezuelan  Equine  Encephalitis  LOD 


Concentration 

(PFU/mL) 

Total  PFU 

Total  GE 

Results 

(Positives/Total  Runs) 

1.00x10s 

2.50xl07 

2.50xl07 

0/1 

Vendor  Stated  LOD:  N/A 


Clostridium  botulinum  Type  A  toxin 
Protein  toxin 

Active  BoNT  A  was  diluted  serially  in  10-fold  increments  into  buffer  (PBS  +  0.1%  Tween-20)  and  tested 
beginning  at  lOOng/mL.  Testing  progressed  as  described  in  Figure  6.  BoNT  A  was  detected  in  neither  the 

130 

Approved  for  Unlimited  Distribution 
Unclassified 


ANTIBODY  TECHNOLOGIES 


Seattle  Sensor  Systems 

CORPORATION 

100  nor  l,000ng/mL  sample.  The  LOD  of  the  device  was  determined  to  be  greater  than  1.00xl03ng/mL. 
Testing  of  concentrations  above  this  level  was  not  possible  due  to  the  large  sample  volume  required. 


Table  58.  Clostridium  botulinum  Type  A  LOD 


Concentration  (ng/mL) 

Total  Toxin  (ng) 

Results 

(Positives/Total  Runs) 

l.OOxlO2 

2.50X101 

0/1 

l.OOxlO3 

2.50xl02 

0/1 

Vendor  Stated  LOD:  N/A 


Multiplex  Detection 

Multiplexing  was  not  performed,  as  the  instrument  was  capable  of  only  analyzing  a  single  reference  and 
test  channel  at  this  phase  of  development. 

Discussion 

Call  Assignments 

Systems  calls  were  made  from  interpretation  of  SPR  signal  on  a  gold  surface  coated  with  antibody. 
Bound  antigen  in  a  test  channel  provides  an  increase  in  signal  and  deflection  from  unbound  antibody 
signal  alone  in  a  reference  channel.  Calls  include  "detected/'  "warning,"  and  "no  call,"  with  slope  and  R2 
values  given  as  fitness  metrics.  The  significance  of  such  metrics  is  not  known.  Frequent  "bad  call"  system 
messages  were  seen,  but  referred  to  as  normal  by  vendor  personnel.  The  instrument's  operating  system 
records  and  displays  real-time  sensor  data,  but  this  data  does  not  provide  a  system  call  until  the  data  is 
analyzed  after  the  run  with  a  separate  software  package. 

Assay  Sensitivity 

The  assay  format  is  SPR  detection  of  a  single  antibody  capture  assay.  No  attempt  was  made  to  increase 
signal,  sensitivity,  or  specificity  of  the  assay  using  a  secondary  antibody  with  an  SPR  enhancer,  such  as 
silver  nanoparticles,  despite  the  fact  that  the  vendor  does  have  experience  with  such  systems. 

Sensitivity  of  the  system  did  not  detect  any  of  the  analytes  in  a  meaningful  range,  with  bacterial  CFU 
LODs  of  >106,  viral  PFU  LODs  of  >108,  and  a  complete  inability  to  detect  BoNT  A  at  any  concentration 
tested.  The  sensitivity  was  so  poor  that  additional  testing  with  stock  concentrations  was  considered 
prohibitively  expensive  and  functionally  irrelevant. 

Sensitivity  of  the  system  could  potentially  be  increased  with  better  utilization  of  sample,  as  only  one 
quarter  to  one  half  of  the  total  500-1000pL  was  contained  in  the  sample  fluidics  loop  for  analysis. 
However,  even  a  two-  to  four-fold  increase  in  signal  by  increasing  the  amount  of  accessible  sample 
would  be  insignificant  without  substantial  improvement  in  assay  design. 
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MOBILE  LABORATORY  ASSESSMENT 
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Selection  of  Systems 

At  the  end  of  the  laboratory  assessment  subject  matter  experts  (SME)  evaluated  the  instruments  for 
suitability  in  the  field  assessment  phase.  Factors  considered  to  determine  the  appropriateness  for 
instrument  inclusion  in  the  field  assessment  included,  but  were  not  limited  to: 

•  Ease  of  use 

•  Ease  of  result  viewing 

•  East  of  result  interpretation 

•  Training  simplicity 

•  Safety 

•  Cleaning 

Operators 

Two  members  of  the  20th  Support  Command  (SUPCOM),  based  at  Aberdeen  Proving  Ground,  MD 
performed  the  mobile  laboratory  assessment.  These  operators  had  extensive  experience  using  analytical 
equipment  to  detect  biological  agents  and  required  minimal  training  on  the  equipment  prior  to  testing. 

Mobile  Laboratory 

The  20th  SUPCOM  employs  the  HMEL  for  their  mission.  The  HMEL,  which  has  bench  space,  power 
supply,  and  a  temperature  controlled  environment  for  multiple  instruments  to  run  concurrently,  was 
utilized  for  this  mobile  assessment.  The  staff  was  supplied  with  necessary  equipment  and  reagents  to 
perform  the  assessment. 

Sample  Construction  and  Analysis 

The  mobile  laboratory  assessment  focused  on  determining  the  instruments'  capacity  to  be  run  by  typical 
operators  in  a  mobile  laboratory  environment.  Therefore,  sample  preparation  by  the  operators  was  kept 
to  a  minimum.  Systems  that  analyze  nucleic  acids  utilized  either  the  inactivated  agent,  which  is 
processed  by  integrated  sample  preparation  capability  in  the  instrument  prior  to  analysis  or  by  an 
affiliated  sample  dilution  or  processing  apparatus,  or  purified  DNA  via  a  Qiagen  DNeasy  Kit.  Instruments 
that  utilized  antibody-based  detection  technologies  received  samples  containing  whole  inactivated 
biological  agent  in  pristine  buffer.  Samples  were  prepared  at  a  concentration  of  ten  times  the  LOD 
determined  in  the  Laboratory  Assessments  for  each  instrument.  Each  two-man  team  of  operators 
received  a  single  positive  sample  per  instrument.  The  operators  were  able  complete  assessments  for 
four  instruments  each  day.  The  operators  utilized  the  Operational  Assessment  Table  (Appendix  D)  to 
capture  information  and  opinions  on  the  instruments,  in  addition  to  the  outcomes  of  sample  analyses. 

Results 

Operational  mobile  laboratory  assessments  culminated  in  the  ranking  of  each  system  based  on  six 
specific  attributes:  ease  of  use;  ease  of  result  (viewing  data);  ease  of  result  (interpretation  of  data); 
supporting  documentation;  training  simplicity;  safety;  and  cleaning/maintenance.  These  rankings  were 
quantified  using  a  weighted  scale  to  provide  a  final  score  for  field  operations  based  on  operator  input. 
Each  attribute  was  ranked  by  the  operator  in  one  of  five  categories:  excellent  (five  points);  good  (four 
points);  fair  (three  points);  poor  (two  points);  and  lowest  (one  point).  Each  of  the  operators  assigned  one 
rank  to  each  attribute. 

Using  the  point  system  outlined  above,  a  numerical  value  was  assigned  that  quantifies  the  user  opinion 
of  mobile  laboratory  operations  for  each  system.  The  number  presented  is  the  percent  score  the  system 

133 

Approved  for  Unlimited  Distribution 
Unclassified 


received  in  each  attribute.  For  example,  the  T-COR  4  received  19  of  the  possible  20  points  for  ease  of 
use,  thus  it  received  a  95.  The  overall  score  from  the  T-COR  4  is  a  94,  resulting  in  an  excellent  rating. 


Table  59.  Overall  Mobile  Laboratory  system  scores 


Technology 

System 

Ease  of  Use 

Ease  of 

Result  Viewing 

Ease  of  Result 

Interpretation 

Training 

Simplicity 

Safety 

Cleaning 

Overall 
Score  (%) 

BioFire  FilmArray 

100 

100 

100 

100 

100 

100 

100 

BioFire  RAZOR  EX 

90 

100 

100 

100 

100 

100 

98 

Nucleic-Acid 

Epistem  Genedrive 

70 

90 

90 

100 

70 

100 

87 

IQuum  Liat 

100 

100 

100 

100 

100 

100 

100 

Tetracore  T-COR  4 

90 

100 

100 

90 

100 

100 

97 

ANP  NIDS 

100 

90 

90 

100 

100 

100 

97 

Luminex  MAGPIX 

60 

80 

80 

90 

70 

90 

78 

Antibody 

MSD  Cartridge  Reader 

100 

100 

100 

100 

100 

100 

100 

Research  Int'l  RAPTOR 

50 

80 

80 

70 

60 

70 

68 

Sandia  Nat'l  Lab  SpinDx 

60 

60 

60 

100 

100 

100 

80 

Seattle  Sensors  SPIRIT 

80 

80 

80 

90 

80 

80 

82 

ND  -  Not  Determined.  The  Mobile  laboratory  operators  did  not  evaluate  all  systems  supporting  documentation 


Table  60.  BioFire  FilmArray 


Test  Score 

Aurmute 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

2  of  2 

Ease  of  Result  Viewing 

4  of  4 

Ease  of  Result  Interpretation 

4  of  4 

Training  Simplicity 

2  of  2 

Safety 

2  of  2 

Cleaning/Maintenance 

1  of  1 

Operator  Comments: 

•  Sharps  are  ok. 

•  Biosafety  cabinet  for  setup. 

•  1  hour  is  good  for  mobile. 

•  Less  prep  than  JBAIDS  (JBAIDS,  1-2-3  kit  extraction)  historically. 

•  No  toxin. 

•  Prefer  1  sample,  full  panel. 

•  Reagents  expired  12/16/12,  system  should  have  caught  that. 
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Table  61.  BioFire  RAZOR  EX 


Test  Score 

AlulDUie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  2 

lof  2 

Ease  of  Result  Viewing 

2  of  2 

Ease  of  Result  Interpretation 

2  of  2 

Training  Simplicity 

2  of  2 

Safety 

2  of  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 

•  Like  the  color  coding. 

•  Chart  tells  you  what's  in  each  well. 


Table  62.  Epistem  Genedrive 


Test  Score 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

lof  2 

1  of  2 

Ease  of  Result  Viewing 

1  of  2 

lof  2 

Ease  of  Result  Interpretation 

lof  2 

lof  2 

Training  Simplicity 

2  of  2 

Safety 

lof  2 

lof  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 

•  Bubbles  were  observed  in  each  lane  of  cartridge. 

•  PCR  pellet  came  out  because  of  static. 

•  Possibility  of  cross  contamination  with  the  card. 

Table  63.  IQuum  Liat 

•i 

Test  Score 

Attribute 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

2  of  2 

Ease  of  Result  Viewing 

2  of  2 

Ease  of  Result  Interpretation 

2  of  2 

Training  Simplicity 

2  of  2 

Safety 

2  of  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 

•  Like  this  one  for  a  presumptive  sample. 

•  Not  enough  variability  for  an  unknown. 

•  Refrigeration  is  not  an  issue  (they  have  M1M) 
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Table  64.  Tetracore  T-COR  4 


Test  Score 

AlulDUie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  2 

lof  2 

Ease  of  Result  Viewing 

2  of  2 

Ease  of  Result  Interpretation 

2  of  2 

Training  Simplicity 

1  of  2 

lof  2 

Safety 

2  of  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 

•  Cycles  ended  at  different  times. 

•  Presumptive  ID. 

•  More  steps  than  the  Liat,  with  the  same  results. 

•  Real  time  is  good,  but  they  would  still  wait  on  the  negative. 


Table  65.  ANP  NIDS  SARIN 


Test  Score 

AlulDUie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

2  of  2 

Ease  of  Result  Viewing 

lof  2 

lof  2 

Ease  of  Result  Interpretation 

lof  2 

lof  2 

Training  Simplicity 

2  of  2 

Safety 

2  of  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 

•  NIDS  is  used  in  presumptive  ID  by  a  different  group. 

•  Mobile  is  used  more  for  a  definitive  ID. 


Table  66.  Luminex  MAGPIX 


Test  Score 

Attribute 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

2  of  2 

Ease  of  Result  Viewing 

2  of  4 

2  of  4 

Ease  of  Result  Interpretation 

2  of  4 

2  of  4 

Training  Simplicity 

lof  2 

lof  2 

Safety 

lof  2 

lof  2 

Cleaning/Maintenance 

lof  2 

lof  2 

Operator  Comments: 

•  Like  this  one  as  a  screening  tool. 

•  Refrigeration  is  not  an  issue  (they  have  MIM) 
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Table  67.  MSP  Cartridge  Reader 


Test  Score 

Attribute 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

2  of  2 

Ease  of  Result  Viewing 

2  of  2 

Ease  of  Result  Interpretation 

2  of  2 

Training  Simplicity 

2  of  2 

Safety 

2  of  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 

•  Superior  system. 

•  Possible  substitute  for  M1M. 

•  Low  handling. 

•  Favorite  of  the  antibody  systems. 

•  Easy  to  use. 

•  Low  possibility  of  cross  contamination. 

•  30  min  run  is  good. 

•  Sound  technology. 

•  Sensitivity  is  not  a  driving  force. 


Table  68.  Research  International  RAPTOR  Plus 


•i 

Test  Score 

Attribute 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  2 

1  of  2 

Ease  of  Result  Viewing 

lof  1 

Ease  of  Result  Interpretation 

lof  1 

Training  Simplicity 

lof  2 

lof  2 

Safety 

2  of  2 

Cleaning/Maintenance 

lof  2 

lof  2 

Operator  Comments: 

•  No  auto-prompt. 

•  Does  not  allow  you  to  insert  the  coupon  incorrectly. 

•  When  cap  was  removed,  there  was  splashing  of  a  liquid. 

•  They  wouldn't  be  able  to  reuse  the  vials. 

•  Only  tests  for  4. 

•  Waste  design  -  a  lot  of  bags  -  or  clean  bags. 
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Table  69.  Sandia  National  Laboratory  SpinDX 


Test  Score 

Aiinouie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  1 

Ease  of  Result  Viewing 

1  of  1 

Ease  of  Result  Interpretation 

1  of  1 

Training  Simplicity 

1  of  1 

Safety 

1  of  1 

Cleaning/Maintenance 

1  of  1 

Operator  Comments: 

•  Can't  run  on  power  cord  at  all  because  the  cord  prevents  the  lid  from  going  down  &  the 


system  turning  on. 

•  Centrifuge  tube  to  make  a  pellet,  difficult  for  pipet  tip  to  pick  up  3  pi  of  prepared  sample  & 
reagents. 

•  "Play"  button  in  software  not  obvious. 

•  Started  taking  baseline  when  closed  lid  and  did  not  wait  for  software. 

•  Channels  1-5  gave  no  baseline. 


Table  70.  Seattle  Sensors  SPIRIT 


Test  Score 

Aiinouxe 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  2 

1  of  2 

Ease  of  Result  Viewing 

2  of  2 

Ease  of  Result  Interpretation 

2  of  2 

Training  Simplicity 

1  of  2 

1  of  2 

Safety 

2  of  2 

Cleaning/Maintenance 

2  of  2 

Operator  Comments: 


•  Sample  volume  is  unrealistic  (lmL). 
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Selection  of  Systems 

During  and  at  the  conclusion  of  the  laboratory  assessment,  operators  and  SMEs  evaluated  the 
instruments  for  suitability  in  the  field  test  assessment  phase.  Factors  that  determined  the 
appropriateness  for  instrument  inclusion  in  the  field  assessment  included,  but  are  not  limited  to: 

•  Size 

•  Battery  Power 

•  Sample  Preparation  Requirements 

•  Reagent  Stability 

•  End  User  Needs 

Operators  and  Testing  Site 

The  operators  for  the  man-portable/handheld  instrument  field  test  assessments  were  members  of  the 
56th  Chemical  Reconnaissance  Detachment  (CRD)  5th  Special  Forces  Group  and  the  Army  22nd  Chemical 
Battalion.  These  operators  had  less  extensive  experience  using  analytical  equipment  to  detect  biological 
agents  and  therefore  underwent  one  day  training  on  the  candidate  instruments  prior  to  performing  this 
assessment.  Testing  was  performed  at  Skippers  Point  Training  Area  at  the  Aberdeen  Proving  Ground, 
Edgewood  Area,  MD.  This  test  site  is  a  housing  duplex  with  surrounding  yard  that  is  currently  used  for 
CBRNE  training  exercises  by  the  Advanced  CBRNE  Training  Team  (See  Figures  7  and  8). 


Figure  7.  Members  of  the  56th  CRD  obtain 
samples  at  Skippers  Point  training  facility. 


Figure  8.  Members  of  the  56  CRD  prepare 
samples. 


Sample  Construction  and  Analysis 

The  field  assessment  focused  on  determining  the  instruments'  capacity  to  be  run  by  typical  operators  in 
a  non-laboratory  environment.  Sample  preparation  by  the  operators  was  kept  to  a  minimum.  Systems 
that  analyzed  nucleic  acids  utilized  either  the  inactivated  agent,  which  is  processed  by  integrated  sample 
preparation  capability  in  the  instrument  prior  to  analysis  or  by  an  affiliated  sample  dilution,  processing 
apparatus,  or  purified  DNA  via  Qiagen  DNeasy  Kit.  Instruments  that  utilized  antibody-based  detection 
technologies  received  samples  containing  whole  inactivated  biological  agent  in  pristine  buffer.  Samples 
were  prepared  at  a  concentration  of  ten  times  the  LOD  as  determined  in  the  Laboratory  Assessments  for 
each  instrument.  Each  two-man  team  or  single  operator  received  a  single  positive  sample  per 
instrument.  The  operators  utilized  the  Operational  Assessment  Table  to  capture  information  and 
opinions  on  the  instruments,  in  addition  to  the  outcomes  of  sample  analyses  (see  Appendix  D). 
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Results 

Operational  field  assessments  culminated  in  the  ranking  of  each  system  based  on  seven  specific 
attributes:  ease  of  use;  ease  of  result  viewing;  ease  of  result  interpretation  of  data;  supporting 
documentation;  training  simplicity;  safety;  and  cleaning/maintenance.  These  rankings  were  quantified 
using  a  weighted  scale  to  provide  a  final  score  for  field  operations  based  on  operator  input.  Each 
attribute  was  ranked  by  the  operator  in  one  of  five  categories:  excellent  (five  points);  good  (four  points); 
fair  (three  points);  poor  (two  points);  and  lowest  (one  point).  Each  of  the  operators  assigned  one  rank  to 
each  attribute. 

Using  the  point  system  outlined  above,  a  numerical  value  that  quantifies  the  user  opinion  of  field 
operations  for  each  system  was  assigned.  The  number  presented  is  the  percent  score  the  system 
received  in  each  attribute.  For  example,  the  Liat  received  29  of  the  possible  30  points  for  ease  of  use, 
thus  it  received  a  97.  The  overall  score  for  the  Liat  is  a  96  resulting  in  an  excellent  rating. 


Table  71.  Overall  handheld/man-portable  system  ratings 


System 

Ease 

of  Use 

Ease  of 

Result 

Viewing 

Ease  of  Result 
Interpretation 

Supporting 

Documents 

Training 

Simplicity 

Safety 

Cleaning 

Overall 

Liat 

97 

100 

100 

90 

100 

100 

80 

96 

Cartridge  Reader 

100 

93 

100 

100 

100 

100 

100 

99 

Genedrive 

73 

90 

53 

60 

87 

97 

90 

79 

NIDS 

100 

93 

93 

100 

100 

83 

100 

95 

RAPTOR 

60 

90 

90 

90 

95 

80 

75 

82 

RAZOR  EX 

80 

93 

93 

90 

93 

93 

75 

89 

T-COR  4 

97 

97 

97 

100 

100 

93 

90 

96 
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Table  72.  BioFire  Diagnostics  RAZOR  EX  ratings 


Lowest 


Ease  of  Use 


2  of  6 


Good 


2  of  6 


Excellent 


2  of  6 


Ease  of  Result  Viewing 


2  of  6 


4  of  6 


Ease  of  Result  Interpretation 


2  of  6 


4  of  6 


Supporting  Documentation 


2  of  4 


2  of  4 


Training  Simplicity 


2  of  6 


4  of  6 


Safety 


2  of  6 


4  of  6 


Cleaning/Maintenance 


1  of  4 


3  of  4 


Operator  Comments 

Manipulation  of  ticket  reduced  ease  of  use  from  Excellent  to  Good. 

Difficulty  using  barcode  scanner.  Suggest  a  device  on  barcode  scanner  for  positioning  assay  at  correct 
distance,  orientation,  and  angle  for  scanning. 

Comb  could  be  stiffer  because  tool  on  end  gets  damaged  before  all  plungers  twisted. 

Assay  boxes  tough  to  get  into  with  multiple  pair  of  gloves. 

Device  needs  to  give  immediate  confirmation  that  assay  has  begun  running. 

Needs  an  easy  to  find  progress  window. 

Carry  strap  gets  in  way  when  trying  to  scan  barcode. 

Screen  hard  to  read,  even  with  the  Brightness  button. 

Progress  screen  should  be  the  default  choice. 

Troubleshooting  section  was  in-depth. 

Too  many  crevices  for  particles  to  get  into.  A  Hypewipe  could  not  get  into  the  crevices. 

Sample/buffer  ports  are  too  hard  to  see. 

Audible  signal  alert  that  successful  scanning  of  the  barcode  would  be  useful. 

Cardboard  insert  in  assay  kit  exchanged  for  a  foam  insert  or  other  device  that  would  hold  bottles  tighter, 
allowing  one-handed  opening  of  bottles  would  be  an  improvement. 
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Table  73.  Epistem  Genedrive  ratings 


Test  Score 

Arcnpuie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

2  of  6 

4  of  6 

Ease  of  Result  Viewing 

3  of  6 

3  of  6 

Ease  of  Result  Interpretation 

2  of  6 

2  of  6 

2  of  6 

Supporting  Documentation 

4  of  4 

Training  Simplicity 

2  of  6 

4  of  6 

Safety 

1  of  6 

5  of  6 

Cleaning/Maintenance 

1  of  4 

3  of  4 

Operator  Comments 

•  The  manual  should  go  over  how  to  interpret  and  view  results. 

•  Instrument  as  it  is  would  not  be  taken  "down  range";  sample  test  cartridge  would  be  loaded  down 
range,  de-conned,  brought  back  to  a  cold  zone,  and  ran  on  an  instrument. 

•  Very  impressed  with  the  size. 

•  Device  does  not  allow  input  of  sample  information. 

•  Operators  noted  that  there  were  too  many  fine  movements  required  for  use  in  a  tactical  situation  and 
that  set-up  took  too  long.  Device  would  not  be  used  in  field  because  of  limited  time  on  target.  Operators 
noted  there  were  too  many  accessories. 

•  Operator  1  had  trouble  pushing  button  of  Genedrive  to  start  sample  run  while  wearing  MOPP  gear. 

Used  one  of  the  bead  tubes  to  push  button. 

Operator  2  dropped  assay  cartridges  when  setting  up  reactions.  In  this  scenario,  operators  had  unlimited 
supply  of  reagents  but  this  may  be  an  issue  in  the  field  with  limited  resources. 

Table  74.  IQuum  Liat  ratings 
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Table  75.  Tetracore  T-COR  4  ratings 


Test  Score 

AlulDUie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  6 

5  of  6 

Ease  of  Result  Viewing 

1  of  6 

5  of  6 

Ease  of  Result  Interpretation 

1  of  6 

5  of  6 

Supporting  Documentation 

4  of  4 

Training  Simplicity 

6  of  6 

Safety 

1  of  6 

5  of  6 

Cleaning/Maintenance 

1  of  4 

3  of  4 

Operator  Comments 

•  Backlight  on  instrument  could  be  brighter. 

•  Centrifuge  was  still  spinning  when  opened,  safety  concern. 

•  Instruments  looks  like  it  could  go  down  range  but  would  not  be  eligible  because  of  the  inability  to  be 
decontaminated. 

•  Real  time  was  excellent. 

•  Results  were  easily  interpreted. 

•  Operator  commented  that  a  handle  would  be  preferable. 

Table  76.  ANP  NIPS  SAR  III  ratings 


Test  Score 

Auriouie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

6  of  6 

Ease  of  Result  Viewing 

2  of  6 

4  of  6 

Ease  of  Result  Interpretation 

2  of  6 

4  of  6 

Supporting  Documentation 

4  of  4 

Training  Simplicity 

6  of  6 

Safety 

1  of  6 

1  of  6 

4  of  6 

Cleaning/Maintenance 

4  of  4 

Operator  Comments 

•  Monitor  easy  to  read. 

•  Good  contrast. 

•  Like  that  it  is  small  and  portable. 

•  Reader  makes  interpretation  of  HHA  consistent  from  user  to  user.  Buttons  could  be  a  little  bigger. 

•  Users  are  concerned  with  chance  of  device  becoming  contaminated  with  agent  since  the  cartridge  is 
inserted  into  the  device  without  any  cover  over  the  sample  pad. 

•  User  noted  that  device  is  small  enough  to  fit  into  pouch  on  body  armor. 
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Table  77.  MSP  Cartridge  Reader  ratings 


Test  Score 

AlulDUie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

6  of  6 

Ease  of  Result  Viewing 

2  of  6 

4  of  6 

Ease  of  Result  Interpretation 

6  of  6 

Supporting  Documentation 

4  of  4 

Training  Simplicity 

6  of  6 

Safety 

6  of  6 

Cleaning/Maintenance 

4  of  4 

Operator  Comments 

•  Instrument  does  not  have  a  battery,  obviously  not  for  the  field. 

•  Run  time  is  very  short  compared  to  others. 

•  Ability  to  test  for  multiple  targets  is  very  good. 

•  Operators  concerned  whether  units  build  static  charge  which  could  spark  a  chemical  or  explosive  agent 
in  immediate  area. 

•  Very  little  time  required  to  set  up  equipment. 

Table  78.  Research  International  RAPTOR  ratings 


Test  Score 

AlulDUie 

Lowest 

Poor 

Fair 

Good 

Excellent 

Ease  of  Use 

1  of  4 

1  of  4 

2  of  4 

Ease  of  Result  Viewing 

1  of  4 

1  of  4 

Ease  of  Result  Interpretation 

1  of  4 

1  of  4 

Supporting  Documentation 

2  of  4 

2  of  4 

Training  Simplicity 

1  of  4 

3  of  4 

Safety 

2  of  4 

2  of  4 

Cleaning/Maintenance 

1  of  4 

3  of  4 

Operator  Comments 

•  Two  operators  were  not  able  to  run  a  sample  due  to  errors;  therefore,  no  rating  for  ease  of 
interpretation  was  given. 

•  Set  up  was  too  labor  intensive  and  small  parts  could  get  lost. 

•  Tubing  was  too  short  and  dexterity  was  a  major  issue. 

•  Color  confusion  was  an  issue.  Colors  need  to  match  or  not  be  used. 

•  Stainless  steel  adapters  for  the  reagent  vials  could  potentially  puncture  their  gloves. 

•  Decon  would  be  difficult  with  this  device  because  sample  interacts  with  interior  of  device. 

•  Device  was  too  heavy. 

•  Suggestions  for  improvement  include:  adding  stiffeners  to  the  end  of  the  tubing,  color  coding  the 
spigots,  making  colors  of  reagent  vials  match  the  color  of  the  corresponding  tubing  assembly/cap  device. 
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SUMMARY  OF  SYSTEM  SCORES 


Approved  for  Unlimited  Distribution 
Unclassified 


146 


Overall 

The  relative  overall  scores  represent  the  cumulative  assessment  scores  that  were  adjusted  so  the 
highest  scoring  system,  the  Liat,  had  a  Relative  Overall  score  of  100.  The  Relative  Overall  scores  of  these 
systems  are  most  notable  for  aggregating  around  the  80th  percentile.  In  other  words,  no  particular 
instrument  stood  out  among  the  rest  as  being  vastly  superior  or  was  a  relative  failure  when  considering 
all  aspects  of  biological  agent  identification.  The  IQuum  Liat  was  the  highest  scoring  nucleic  acid-based 
biological  identifier,  while  the  ANP  NIDS  was  the  highest  scoring  antibody-based  system  according  to  the 
performance  criteria  and  scoring  algorithm. 


OVERALL 


Figure  9.  Overall  summary  of  system  scores 
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Singleplex  Target  Sensitivity 

As  expected,  two  nucleic  acid-based  systems,  FilmArray  and  Liat,  scored  the  best  for  single  target 
detection  sensitivity  in  spite  of  being  "penalized"  for  not  being  capable  of  detecting  the  toxin  BoNT  A. 
The  RAPTOR  and  SPIRIT  were  lowest  scoring  systems,  mainly  due  to  their  inability  to  detect  all  targets 
consistently  at  relatively  low  LODs.  Of  particular  note  when  comparing  sensitivities,  the  sample  size  that 
is  actually  analyzed  varies  from  system  to  system.  For  instance,  the  T-COR  4  only  requires  1  uL  of  sample 
per  assay  tube,  while  the  RAPTOR  requires  l-2mL  of  sample  per  assay.  The  Singleplex  Sensitivity  was 
determined  using  concentration  of  target  within  the  sample,  although  the  actual  volume  of  sample 
analyzed  was  different. 


SINGLEPLEX  TARGET  SENSITIVITY 


Figure  10.  Summary  of  Singleplex  Sensitivity  scores 
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Multiplex  Target  Sensitivity 

In  general,  the  instruments  retained  nearly  the  same  sensitivity  for  singleplex  targets  when  multiple 
targets  were  in  the  sample,  although  instruments  without  the  full  complement  of  multiplex  capabilities 
had  lower  possible  scores  in  this  category.  The  Liat  and  FilmArray  received  relatively  high  scores  for 
having  very  high  sensitivity  for  the  targets  they  could  identify.  Three  instruments  (Genedrive,  SpinDx, 
and  T-COR  4)  received  scores  of  zero  for  multiplex  sensitivity  because  they  were  unable  to 
simultaneously  run  assays  for  multiple  targets.  Conversely,  some  instruments  could  perform  duplex 
(Liat,  NIDS,  RAZOR  EX),  triplex  (NIDS),  or  4-plex  (FilmArray),  or  the  full  5-plex  analyses  (Cartridge  Reader, 
MAGPIX).  The  RAZOR  EX  had  the  capability  of  detecting  10  targets  using  the  "10-Pouch"  assay  kit; 
however,  only  two  of  the  assessment  targets  were  part  of  the  "10-Pouch"  kit. 

MULTIPLEX  TARGET  SENSITIVITY 


Figure  11.  Summary  of  Multiplex  Sensitivity  scores 
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Multiplex  Capability 

The  Multiplex  Capability  attribute  was  comprised  of  the  scores  for  "Number  of  reportable  agents  per 
run"  and  "Number  of  individual  targets  per  test"  categories.  The  MAGPIX  had  the  highest  score  for  this 
attribute  because  the  highly  multiplex  beads  allow  up  to  50  targets  to  be  simultaneously  detected.  In 
contrast,  the  SPIRIT,  as  configured,  could  analyze  only  one  sample  at  a  time  for  one  target.  The 
FilmArray  had  multiple  targets  per  test  for  B.  anthrocis  (i.e.,  a  chromosomal  and  two  plasmid  targets) 
and  other  agents,  while  all  other  instruments,  as  tested,  were  capable  of  only  one  target  per  test  agent. 

MULTIPLEX  CAPABILITY 


Figure  12.  Summary  of  Multiplex  Capability  scores 
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Assay  Flexibility 

The  Assay  Flexibility  attribute,  intended  to  capture  the  ease  with  which  new  assays  could  be 
incorporated  to  the  system,  was  comprised  of  the  performance  criteria  scores  for  "Number  of  sources  of 
consumables  or  self-designed  assays"  and  "New  assay  integration"  categories.  The  MAGPIX  and  T-COR  4 
received  maximum  scores  due  to  their  ability  to  easily  integrate  new  antibodies  or  primers,  respectively, 
into  new  assays.  Meanwhile,  the  FilmArray,  RAZOR  EX,  and  Liat  were  somewhat  closed  systems  in  that 
users  would  have  difficulties  and/or  delays  in  adding  new  assay  targets  to  their  menu. 


ASSAY  FLEXIBILITY 


Figure  13.  Summary  of  Assay  Flexibility  scores 
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Batch  Size 

The  Batch  Size  attribute  was  defined  by  the  following,  "If  looking  for  a  single  agent,  the  number  of 
samples  that  can  be  processed  at  the  same  time  by  one  analyzer."  The  MAGPIX  had  the  highest  score  for 
this  attribute  because  the  plate  format  allows  for  up  to  96  samples  to  be  processed  at  the  same  time. 
Most  systems,  as  currently  configured,  were  only  able  to  analyze  a  single  sample  at  one  time. 


BATCH 


Figure  14.  Summary  of  Batch  Size  scores 
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Run  Time 

The  Run  Time  attribute  is  a  combination  of  system  warm-up  time,  sample  preparation  time,  and  time  for 
analysis  of  the  sample.  Several  systems  scored  moderately  high  as  they  had  combinations  of  low  or  no 
warm-up  requirement,  no  sample  preparation,  and  analysis  of  less  than  35  minutes.  The  NIDS  scored 
highest  as  there  is  no  warm-up  required,  the  sample  may  be  directly  applied,  and  the  run  is  complete 
within  16  minutes.  The  FilmArray,  Genedrive,  and  T-COR  4  had  mediocre  scores  as  PCR  requires  a 
relatively  longer  time  to  perform.  Although  both  the  Liat  and  RAZOR  EX  utilize  PCR,  these  systems  were 
able  to  shave  the  analysis  time  for  higher  scores.  The  MAGPIX  required  multiple  incubations  which 
pushed  the  total  analysis  time  over  two  hours. 


RUNTIME 


Figure  15.  Summary  of  Run  Time  attribute  scores 
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Size 

The  Size  score  was  determined  by  the  system  footprint,  or  area  required  to  operate  the  system  as  well 
as  the  system  weight.  Several  systems  achieved,  or  nearly  achieved,  the  goals  of  being  less  than  two 
pounds  and  1.5  square  feet.  These  systems  are  most  appropriate  for  handheld  or  man-portable  usage. 
When  including  other  characteristics  such  as  battery  power  and  no  ancillary  equipment,  the  Genedrive 
and  NIDS  best  fit  the  handheld  instrument  requirements.  The  MAGPIX  and  FilmArray  scored  lowest  in 
size  attributes;  however,  the  MAGPIX  is  presently  a  research-oriented  instrument  and  the  FilmArray  a 
clinical  diagnostic  instrument,  so  they  are  suitable  only  for  the  mobile  laboratory. 

SIZE 


100  100 


Figure  16.  Comparison  of  Size  scores 
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Power 

The  Power  attribute  noted  (1)  the  ability  of  a  system  to  run  by  battery  power  and  (2)  the  anticipated 
power  draw  (in  Watts)  of  the  instrument.  The  high  scores  in  this  category  denoted  instruments  that 
were  battery-powered,  as  this  was  weighted  high  for  end-users  in  field  situations.  The  MAGPIX  and 
FilmArray  systems  scored  lowest  for  Power  attribute  because  they  do  not  utilize  a  battery  and  draw  up 
to  900  Watts. 


POWER 


Figure  17.  Summary  of  Power  scores 
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Assay  Design 

Assay  design  is  a  parameter  that  was  difficult  to  control  across  the  various  platforms  evaluated  in  this 
study.  Materials  from  the  CRP  were  made  available  to  each  performer  so  that  standardized  assays  could 
be  used,  but  this  was  often  not  practical  for  all  participants.  Targets  were  chosen  to  encompass  a  wide 
spectrum  of  etiological  agents  including  bacteria,  viruses,  and  toxins  to  fully  assess  the  devices'  utility  in 
detecting  a  broad  range  of  targets.  Manufacturers,  such  as  ANP,  had  existing  assays  for  their  hand-held 
immunoassay  cartridges  and  NIDS  SAR  III  reader  that  covered  the  agents  chosen  for  this  evaluation. 
Other  manufacturers,  such  as  BioFire  Diagnostics,  had  COTS  assays  for  the  RAZOR  EX  that  did  not  detect 
all  of  the  chosen  agents;  and,  because  of  their  closed  architecture,  an  assay  could  not  quickly  be 
developed.  On  the  other  hand,  companies  including  Mesoscale  Diagnostics  and  Epistem  created  custom 
assays  specifically  for  this  study  using  the  materials  provided.  As  an  added  complication  to  controlling 
assay  design,  two  complementary  technologies  were  evaluated:  direct  immunological  detection  and 
nucleic-acid  amplification.  Throughout  the  evaluation,  efforts  were  made  to  directly  compare  detection 
across  all  platforms  regardless  of  the  technical  divergence.  Rather  than  reporting  merely  the 
concentration  of  agent  required  for  detection,  the  total  amount  of  material  required  was  reported,  and 
in  the  case  of  Epistem  and  Tetracore,  amount  of  purified  DNA  was  converted  to  amount  of  agent. 

Target  Choice 

Target  choice,  especially  for  the  bacterial  pathogens,  has  a  significant  influence  on  assay  sensitivity  and 
specificity.  It  has  become  common  for  assay  developers  to  target  both  plasmids  and  chromosomal  loci  in 
multiplex  assays.  This  provides  organism  identification  as  well  as  confirmation  of  episomal  virulence 
factors  simultaneously.  Plasmids  are  usually  found  in  multiple  copies  per  cell,  depending  upon  their  size 
and  nature  of  their  replication  genetics.  Targeting  such  high-copy  number  loci  can  provide  more 
consistent  and  sensitive  detection,  especially  when  trace  contamination  is  present.  However,  plasmids 
are  often  shared  between  species  through  horizontal  gene  transfer,  which  can  lead  to  equivocal  results 
and  loss  of  specificity.  There  is  an  inherent  trade-off  in  terms  of  sensitivity  and  specificity  when  using 
either  chromosomal  or  plasmid  targets  without  multiplex,  multi-locus  confirmation.  Of  the  technologies 
evaluated,  the  BioFire  Diagnostics  FilmArray  was  the  only  device  for  which  multiplex  assays  amplifying 
more  than  one  target  per  agent  were  available. 

Direct  detection  of  toxins  represents  a  challenge  for  molecular  methods,  relying  on  residual  DNA  to 
provide  signal  for  detection.  Unfortunately,  only  the  crudest  of  toxin  preparations  would  be  expected  to 
reliably  retain  a  molecular  signature  and  the  physical  characteristics  of  the  two  classes  of 
macromolecules  (protein  and  nucleic  acids)  and  the  physical  methods  used  to  purify  them  are  quite 
disparate.  In  this  evaluation,  three  separate  PCR  detection  systems  (BioFire  RAZOR  EX,  BioFire  Film 
Array,  and  Tetracore  T-COR  4)  were  unable  to  detect  the  Botulinum  A  toxin  reference  material.  The  only 
system  not  running  an  immunoassay  to  detect  the  toxin  was  the  Epistem  Genedrive  which  used  a  novel 
fluorogenic  activity  assay  with  a  synthetic  labeled  peptide  target.  Hybrid  detection  systems  such  as  the 
Luminex  MAGPIX/xMAP  technologies  allow  for  detection  of  both  molecular  and  immunological  targets 
although  only  the  immunoassay  component  of  this  system  was  evaluated  in  this  trial. 

Cross  Platform  Comparisons 

As  noted  above,  assay  constraints  (product  specific  assays/gaps  in  assay  availability  for  some  platforms) 
and  systems  with  and  without  on-board  sample  preparation  made  comparisons  of  overall  system 
sensitivity  difficult.  To  compensate  for  this  lack  of  controlled  testing,  every  effort  was  made  to  ascertain 
the  sensitivity  of  each  platform  (hardware,  software,  and  assay)  and  present  the  data  in  a  meaningful 
manner  for  acquisition  managers.  To  this  end,  each  set  of  results  is  presented  in  terms  of  target 
concentration  (CFU/mL,  PFU/mL,  or  ng/mL),  absolute  quantity  per  reaction  (CFU,  PFU,  mass,  or  GE),  and, 

157 


Approved  for  Unlimited  Distribution 
Unclassified 


when  available,  assay  target  in  order  to  control  for  internal/external  sample  preparation  and  disparate 
volumes  used  per  system  run.  All  certified  reference  materials  (organisms,  toxins,  and  DNA/RNA)  were 
provided  by  the  CRP,  and,  where  possible,  CRP  assays  were  employed. 

In  general,  products  fell  into  two  separate  categories  of  maturity.  Most  of  the  systems  had  high 
technology  readiness  and  represented  FDA  approved  COTS  and/or  deployed  DoD  technologies.  A 
minority  subset  of  platforms  was  more  developmental  in  nature  at  much  lower  technology  readiness, 
which  prevented  complete  or,  in  some  cases,  any  testing  of  proposed  capabilities.  While  not  ready  for 
formal  acquisition  programs  (as  indicated  by  the  testing  results),  this  should  not  preclude  these  less 
mature  systems  from  being  reevaluated  after  additional  development  has  taken  place  to  be  considered 
for  future  programs.  Biological  detection  methodologies  are  evolving  very  quickly,  especially  in  the 
miniaturization  of  immunoassays  and  the  speed  and  detection  of  nucleic  acid  amplification.  By 
evaluating  less  mature  systems,  a  technology  readiness  assessment  allows  an  initial  glimpse  at  emerging 
and  future  technologies  and  will  provide  acquisition  managers  with  a  point  of  reference  to  gauge 
improvements  in  performance. 

While  every  effort  was  made  to  benchmark  hardware  functionality  and  sensitivity,  it  was  impossible  to 
completely  decouple  hardware  performance  from  assay  performance.  The  fact  that  some  platforms 
have  closed  architectures  that  cannot  be  quickly  or  easily  adapted  to  third-party  assays  does  represent  a 
weakness  compared  to  others  that  can  quickly  adapt  and  optimize  new  content.  It  is  important  for 
acquisition  managers  to  understand  that  validity  and  performance  of  assays  is  in  no  way  related  to  the 
performance  of  the  underlying  hardware  systems ,  and  that  both  should  be  evaluated  independently 
whenever  possible. 

In  most  cases,  overall  system  sensitivity  was  at  least  slightly  less  than  advertised  by  manufacturers.  This 
is  likely  attributable  by  the  lack  of  uniform  reference  materials  across  the  diagnostics  and  biodefense 
industry  space  and  variability  of  testing  materials  independently  prepared  by  a  large  number  of 
performers.  However,  each  system  was  evaluated  with  the  same  lots  of  reference  materials  (prepared 
where  possible  and  appropriate  under  ISO  Guide  34)  with  testing  performed  in  as  ISO  17025-reference 
laboratory.  While  it  is  possible  to  dispute  minor  differences  in  results  between  a  manufacturer's 
laboratory  evaluation  with  those  presented  here,  the  relative  differences  in  performance  between 
platforms  in  this  evaluation  are  defensible  and  suitable  to  support  early-phase  acquisition  decision¬ 
making. 

Throughput  and  Data  Density 

The  technologies  evaluated  span  a  wide  range  of  throughput  and  data  density,  going  from  single  sample, 
singleplex  detection  to  96-well  plate  format  with  highly  multiplexed  detection.  Most  of  the  systems 
tested  fall  into  the  middle  of  this  performance  spectrum,  providing  intermediate  throughput  with  more 
flexible  assay  and  use-case  options.  Cost,  processing  time,  overall  throughput,  and  veracity  of  results  are 
all  important  criteria  for  instrument  selection  to  match  specific  Concept  of  Operations  (CONOPS). 

Sample  to  Answer 

One  recent  push  for  recent  technology  development  is  the  advent  of  "sample-to-answer"  systems  that 
contain  on-board  sample  preparation  and  can  provide  a  rapid,  automated  diagnostic  result.  Most  of  the 
devices  tested  in  this  evaluation  were  examined  with  this  push  in  mind  and  were  challenged  with  whole 
organism  regardless  of  whether  they  had  on-board  sample  preparation  capabilities.  The  exceptions  to 
this  were  the  Tetracore  T-COR  4  and  Epistem  Genedrive,  both  of  which  were  tested  with  purified  nucleic 
acids.  The  more  advanced  systems  were  able  to  provide  FDA-approved  multiplex  syndromic  panel 
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detection  for  respiratory/gastrointestinal  infections  and  biological  threat  agents.  This  capability  is  both 
impressive  and  important  for  routine  clinical  applications,  but  at  this  phase  of  development  has  a  high- 
cost  per  sample,  low  throughput  (less  than  twenty  samples  per  eight-hour  shift),  and  fixed  assay 
content. 

Batch  processing 

Higher  throughput  and  higher  multiplex  systems  provide  another  facet  of  detection  and  diagnostic 
capabilities.  As  a  rule,  such  systems  require  a  higher  level  of  trained  staff,  additional  ancillary  equipment 
for  sample  preparation/processing,  and  more  sophisticated  laboratory  resources  to  employ.  Straddling 
the  gap  between  research  applications  and  FDA-approved  clinical  use,  they  provide  the  most  flexible 
technology  base  for  emerging  infectious  disease  threats  with  the  lowest  per-sample  costs.  By  combining 
high  levels  of  multiplexing  with  superior  through-put,  these  systems  generate  the  highest  data  density 
and  support  processing  of  large  numbers  of  samples  and  multi-locus  confirmatory  testing. 

Study  Limitations 

The  primary  purpose  of  this  study  was  to  evaluate  the  sensitivity  of  biological  detection  and  diagnostic 
equipment,  by  verifying  vendor  claims  and  providing  baseline  comparisons  between  platforms.  As 
discussed,  it  was  not  possible  in  all  cases  to  eliminate  other  variables  such  as  varying  assay  content  and 
sample  processing  capabilities.  Data  was  presented  in  a  variety  of  formats  to  allow  direct  comparisons 
where  possible,  caveating  differences  of  assay  targets,  sample  volume  utilization,  and  other 
confounding  parameters. 

Assay  specificity  was  not  addressed  in  a  systematic  manner,  only  evaluating  a  total  of  five  viral,  bacterial, 
and  protein  toxin  targets  for  which  assays  were  available.  No  near-neighbor  organisms;  pathogens 
causing  "look-alike"  disease;  host,  vector,  environmental  samples;  or  wide  biodiversity  panels  were 
tested  looking  for  clinical  or  environmental  cross  reactivity.  Any  system  or  assay  testing  that  resulted  in 
a  false  positive  result  with  such  a  small  testing  cohort  should  be  scrutinized  for  assay  or  instrument 
design  flaws  that  would  need  to  be  addressed  before  fielded  use. 

No  specific  recommendations  are  suggested  to  match  technologies  to  CONOPS.  In  each  case,  system 
capabilities,  utility,  and  potential  performance  limitations  have  been  documented  to  enable 
procurement  managers,  end  users,  and  subject  matter  experts  to  use  dependable  test  data  to  provide 
the  best  and  most  appropriate  equipment  for  specific  uses. 
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SUMMARY 

Given  the  broad  range  of  requirements  for  diagnostic  and  environmental  testing,  there  is  no  single 
technology  that  provides  a  comprehensive  solution  for  all  needs.  Routine,  low-throughput  screening  of 
known  threats  is  often  most  cost-effectively  handled  with  rapid,  low  data  density  testing  formats. 
However,  epidemic/pandemic  outbreaks,  wide  area-contaminations,  and  emerging  biological  threats 
require  scalable  technologies  with  the  ability  to  scan  a  wider  array  of  threats. 

High  consequence  actions  such  as  use  of  medical  countermeasures  or  evacuation  of  critical 
infrastructure  require  a  high  burden  of  proof  before  implementation.  Singleplex  detection  of  biological 
threats  is  generally  not  considered  to  be  sufficient  to  meet  the  burden  of  proof  required  for  such 
actions,  nor  to  rule  out  a  potentially  catastrophic  threat  for  fear  that  a  false-negative  result  will  miss  a 
new  or  genetically  diverse  existing  threat. 

A  well-rounded  suite  of  capabilities  is  required  to  meet  the  known  scenarios  for  biological  surveillance 
and  defense.  It  is  important  that  when  specific  applications  and  CONOPS  are  considered,  that 
appropriate  capabilities  are  matched  in  terms  of  costs,  throughput,  data  density,  and  flexibility  of 
architecture. 
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Technology 
Readiness 
Level  (TRL) 

Definition 

ECBC  Biosensors  Test  Bed  Study 

TRL  Interpretations 

Department  of  Defense  Technology 
Readiness  Assessment  Guidance 
Descriptions 

1 

Basic  principles 
observed  and 
reported. 

Published  research  has  identified  the 
principles  that  underlie  this  technology. 
Basic  scientific  research  has  started  to  be 
translated  into  applied  research  and 
development.  Examples  include  scientific 
literature  and  paper  studies  of  a  biological 
identifier  technology's  basic  properties. 

Lowest  level  of  technology  readiness. 
Scientific  research  begins  to  be  translated 
into  applied  research  and  development 
(R&D).  Examples  might  include  paper 
studies  of  a  technology's  basic  properties. 

2 

Technology  concept 
and/or  application 
formulated. 

Basic  biological  identification  technology 
principles  have  been  expressed  in 
practical  applications.  Applications  are 
speculative,  and  there  may  be  no  proof  or 
detailed  analysis  to  support  the 
assumptions.  Examples  include 

technologies  that  are  limited  to  analytical 
studies. 

Invention  begins.  Once  basic  principles  are 
observed,  practical  applications  can  be 
invented.  Applications  are  speculative,  and 
there  may  be  no  proof  or  detailed  analysis 
to  support  the  assumptions.  Examples  are 
limited  to  analytic  studies. 

3 

Analytical  and 
experimental  critical 
function  and/or 
characteristic  proof  of 
concept. 

Active  research  and  development, 
including  analytical  and  laboratory  studies 
to  validate  the  analytical  predictions  of 
separate  elements  of  the  biological 
identifier  technology.  Examples  include 
components  that  measure  parameters  of 
interest  but  are  not  yet  integrated  into  or 
representative  of  the  final  system. 

Active  R&D  is  initiated.  This  includes 
analytical  studies  and  laboratory  studies  to 
physically  validate  the  analytical  predictions 
of  separate  elements  of  the  technology. 
Examples  include  components  that  are  not 
yet  integrated  or  representative. 

4 

Component  and/or 
breadboard  validation 
in  a  laboratory 
environment. 

Basic  biological  identification  technology 
components  are  integrated  to  establish 
feasibility.  Non-optimized  components 
are  assembled  and  operated  together. 
Laboratory  testing  shows  capability  to 
identify  biological  warfare  agent  (BWA) 
surrogates  and/or  biological  simulants. 
Examples  include  integration  of  some 
components  and  "ad  hoc"  hardware  in 
the  laboratory. 

Basic  technological  components  are 
integrated  to  establish  that  they  will  work 
together.  This  is  relatively  "low  fidelity'' 
compared  with  the  eventual  system. 
Examples  include  integration  of  "ad  hoc'' 
hardware  in  the  laboratory. 

5 

Component  and/or 
breadboard  validation 
in  a  relevant 

environment. 

Basic  technology  components  are 
integrated  with  supporting  elements  to 
function  as  a  biological  identifier  in  a 
simulated  environment.  Fidelity  of  the 
technology  components  is  very  high. 
System  successfully  shows  capability  to 
identify  BWA  surrogates  or  biological 
simulants.  Initial  assay  development  and 
sensitivity  testing  occurs. 

Fidelity  of  breadboard  technology  increases 
significantly.  The  basic  technological 
components  are  integrated  with  reasonably 
realistic  supporting  elements  so  they  can  be 
tested  in  a  simulated  environment. 
Examples  include  "high-fidelity''  laboratory 
integration  of  components. 

6 

System  model  or 
prototype 
demonstration  in  a 
relevant  environment. 

Biological  identifier  system  is  well 
advanced  beyond  prototype  and 
demonstrates  technology's  readiness. 
System  and  individual  components  are 
tested  in  a  relevant  environment  for 
successful  BWA  surrogate  or  biological 
simulant  identification.  Optimization  of 
assay  performance  with  system. 

Representative  model  or  prototype  system, 
which  is  well  beyond  that  of  TRL  5,  is  tested 
in  a  relevant  environment.  Represents  a 
major  step  up  in  a  technology's 
demonstrated  readiness.  Examples  include 
testing  a  prototype  in  a  high-fidelity 
laboratory  environment  or  in  a  simulated 
operational  environment. 

A-2 
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7 

System  prototype 
demonstration  in  an 
operational 
environment. 

Biological  identifier  is  at  the  operational 
stage  and  is  a  commercial-off-the-shelf 
(COTS)  item.  Major  system  components 
and  software  are  in  their  final  design. 
System  operation  is  demonstrated  in 
laboratory  and  field  with  BWA  surrogates 
and/or  biological  stimulants  and  by  field 
operators. 

Prototype  near  or  at  planned  operational 
system.  Represents  a  major  step  up  from 
TRL  6  by  requiring  demonstration  of  an 
actual  system  prototype  in  an  operational 
environment  (e.g.,  in  an  aircraft,  in  a 
vehicle,  or  in  space). 
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FilmArray  (1  of  2) 

Attribute 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Category 

Attribute 

Weight 
(1  to  10) 

Description 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Overall  Score  % 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

500 

7 

70 

5000 

7 

70 

OJD  S3 

c3  2 

H  '■§ 

£  c 

11 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

50 

9 

90 

5 

10 

100 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

Unknown 

0 

0 

1000 

7 

70 

56 

"So  «§ 

.S  1-1 

33 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

1000 

7 

70 

1000000 

4 

40 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

N/A 

0 

0 

0 

0 

0 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

500 

7 

35 

5000 

7 

35 

ox  s 
c3  .2 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

50 

9 

45 

50 

9 

45 

2  « 

.2-  c 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

N/A 

0 

0 

1000 

7 

35 

54 

=  2 

S 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

1000 

7 

35 

1000000 

4 

20 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

N/A 

0 

0 

0 

0 

0 

£  s 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

1  =  0  score; 
2-7 

8-19 

20-34+ 

10 

90 

17 

7 

63 

17 

7 

63 

5  « 

a  a 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

17 

10 

20 

3 

8 

16 

72 

2 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

1 

1 

8 

1 

1 

8 

< 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

1 

1 

Company 

designed 

1 

1 

8 

10 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1  =  0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

g 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

5 

9 

72 

2 

10 

80 

57 

H 

= 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

60 

1 

9 

65 

1 

9 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

Yes.  Sample  is  mixed  with  a  buffer  prior  to  addition  to  assay  pouch. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

About  70  minutes  including  sample  prep,  running  the  pouch,  and  data  interpretation. 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

20 

3 

30 

20 

3 

30 

58 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5 

0  <1.5  sq.  ft. 

10 

90 

1.1 

9 

81 

1.1 

9 

81 

2 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

Logistical  footprint  is  approximately  4.7  sq.  ft. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

Ancillary  equipment  weight  is  approximately  5  lbs. 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

No 

0 

0 

No 

0 

0 

38 

o 

CL 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

90-200 

9 

45 

77 

10 

50 
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FilmArray  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

Final  report  exportable  as  .XPS  only. 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Informational  only 

Assays  shipped  in  kit  of  6  assay  pouches,  less  than  1  sq.  ft.  storage  space  required  per  kit. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

None.  Room  temperature  storage. 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
ml.  per  test 

Informational  only 

Approximately  1  mL  total  waste  remaining  in  sample  and  rehydration  syringes.  Note: 
Canula  is  stainless  steel.  Vendor  recommends  treating  syringe  and  canula  as  sharps 
waste. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$49,500.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$49,500.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$1,100.00 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  BioFire. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

Final  report  exportable  as  .XPS  only. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

No.  Assays  are  self  contained.  Recommend  decontaminating  sample  inlet. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  7. 

GRAND  TOTAL 

48 
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RAZOR  (1  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 

(Wt.x 

Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 

(Wt.x 

Grade) 

Singleplex  Target 
Identification 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

1000 

7 

70 

13000 

5 

50 

24 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

100 

8 

80 

1000 

7 

70 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

N/A 

0 

0 

No  assay 

0 

0 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

1000 

0 

0 

No  assay 

0 

0 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

N/A 

0 

0 

No  assay 

0 

0 

Multiplex  Target 
Identification 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

1000 

7 

35 

26000 

5 

25 

24 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

100 

8 

40 

2600 

7 

35 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

N/A 

0 

0 

No  assay 

0 

0 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

N/A 

0 

0 

No  assay 

0 

0 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

N/A 

0 

0 

No  assay 

0 

0 

Multiplex 

Capability 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

ran 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

10 

5 

45 

10 

5 

45 

43 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

Assay  Flexibility 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

1 

1 

8 

1 

1 

8 

10 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

1 

1 

8 

1 

1 

8 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1=0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Run  Time 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

5 

9 

18 

5 

9 

18 

71 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

5 

9 

72 

5 

9 

72 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

30 

5 

45 

30 

5 

45 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

Yes.  Sample  is  diluted  into  Sample  Dilution  solution  prior  to  loading  pouch. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

Result  automatically  displayed  on  LCD  screen. 

ia 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

11 

7 

70 

11 

7 

70 

84 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

0.31 

10 

90 

0.31 

10 

90 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

None  required. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

N/A 

Power 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

100 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

96 

10 

50 

96 

10 

50 

Approved  for  Unlimited  Distribution 
Unclassified 
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Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

None 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Each  kit  measures  7.125  x  7.25  inches  (0.36  sq.  ft.). 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

None.  Room  temperature  storage. 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Approximately  lmL  liquid  remains  in  syringes  used  for  loading  assay.  Assay  pouch  is 
self-contained.  Sample  dilution  vial  contains  approximately  5mL  liquid. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$38,500.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$38,500.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$200.00 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  by  BioFire  as  necessary. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

Results  viewed  on  screen.  Results  can  be  downloaded  using  a  proprietary  software  and 
laptop. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

External  decontamination  recommended. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  7 

GRAND  TOTAL 

41 

Approved  for  Unlimited  Distribution  B-5 
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Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of  Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Low 

(1-3) 

Medium  (4- 
7) 

High 

(8-10) 

Grade 

Score  (Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt.  x 
Grade) 

Result 

Grade 

Score  (Wt.  x 
Grade) 

Singleplex  Target 
Identification 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfii/ml 

Bacteria:  10e7- 
10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

4200 

6 

60 

2400000 

4 

40 

38 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria:  10e7- 
10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

7200 

6 

60 

18,000 

5 

50 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

6900 

6 

60 

2900000 

4 

40 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

10000 

5 

50 

No  assay 

0 

0 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

No  data 

0 

0 

10  ng/mL 

6 

60 

Multiplex  Target 
Identification 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria:  10e7- 
10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

4200 

6 

30 

N/A 

0 

0 

0 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria:  10e7- 
10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

7200 

6 

30 

N/A 

0 

0 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims:  10e7- 

10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

6900 

6 

30 

N/A 

0 

0 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

10000 

5 

25 

N/A 

0 

0 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

No  data 

0 

0 

N/A 

0 

0 

Multiplex 

Capability 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 
run 

1=0  score;  2- 
7 

8-19 

20-34+ 

10 

90 

4 

2 

18 

1 

0 

0 

2 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for  1 
target 

Score  =  5  for  2 
targets 

Score  =8+  for 
3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

Assay  Flexibility 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for  1 

source 

Score  =  5  for  2 

sources 

Score  =8+  for 
3+  sources 

10 

80 

3+ 

10 

80 

1 

1 

8 

30 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Self 

10 

80 

Company 

designed 

5 

40 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1  =  0  score; 
2-9 

10-50 

50+ 

10 

90 

3 

2 

18 

3 

2 

18 

20 

Run  Time 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

54 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

15 

8 

64 

15 

8 

64 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

60 

2 

18 

60 

2 

18 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

Not  as  advertised,  but  system  was  tested  with  purified  DNA. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

About  70  minutes  including  sample  preparation,  running  the  assay,  and  data 
interpretation. 

ix> 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

1.2 

10 

100 

1.2 

10 

100 

100 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq.  ft. 

0  <1.5  sq.  ft. 

10 

90 

0.5 

10 

90 

0.5 

10 

90 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

1  square  foot 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

0.5  pound 

Power 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

100 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

100 

10 

50 

100 

10 

50 

Approved  for  Unlimited  Distribution 
Unclassified 
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Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of  Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Low  Medium  (4-  High 

(1-3)  7)  (8-10) 

.  Score  (Wt.  x 

Grade  „  ,  . 

Grade) 

.  ,  ,  ,  Score  (Wt.  x 

Claim  Grade  „ 

Grade) 

Score  (Wt.  x 

Result  Grade  _  ,  . 

Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

As  advertised:  2B  BlackBio  BlackLight  paper,  1mm  biopsy  punch,  GE  Ready-to-Go 
PureTaq  PCR  Beads.  As  tested,  did  not  used  paper  or  punch. 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Each  kit  required  <1  square  foot  storage  space. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

As  advertised:  Room  temperature  As  tested:  Primers  required  refrigerator/freezer. 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate  mL 
per  test 

Informational  only 

None 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$4,000.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$4,000.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

Estimated  to  be  $85  dependent  on  volume  purchased. 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  Epistem  or  replacement  unit  provided. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Score  =  1  for  1  target 

As  advertised,  no  exportable  data.  As  tested,  data  was  exported  as  .jpg,  .xls,  and  .txt 
files. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

External  only 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  5. 

GRAND  TOTAL 

42 

Approved  for  Unlimited  Distribution  B-7 
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Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score  (Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt.  x 
Grade) 

Result 

Grade 

Score  (Wt.  x 
Grade) 

Overall 

Score  % 

Singleplex  Target 
Identification 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

9.8 

10 

100 

1000 

8 

80 

64 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

5.4 

10 

100 

6.5 

10 

100 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

250 

8 

80 

2500 

7 

70 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

2100 

7 

70 

2100 

7 

70 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

N/A 

0 

0 

0 

0 

0 

Multiplex  Target 
Identification 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

9.8 

10 

50 

1000 

8 

40 

60 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

5.4 

10 

50 

6.5 

10 

50 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

250 

8 

0 

25000 

6 

30 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

2100 

7 

35 

21000 

6 

30 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

N/A 

0 

0 

0 

0 

0 

Multiplex 

Capability 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

1  =  0  score; 
2-7 

8-19 

20-34+ 

10 

90 

2 

1 

9 

2 

1 

9 

10 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

Assay  Flexibility 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

3+ 

8 

64 

1 

1 

8 

10 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

10 

10 

80 

Company 

designed 

1 

8 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer. 

#  of  samples 

1  =  0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Run  Time 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

2 

9 

18 

2 

9 

18 

80 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

0 

10 

80 

0 

10 

80 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

20 

6 

54 

20 

6 

54 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

None.  Room  temperature  storage. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

1  minute 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

8.3 

8 

80 

8.3 

8 

80 

89 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

<1 

10 

90 

<1 

10 

90 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

No  ancillary  equipment. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

No  ancillary  equipment. 

Power 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

96 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

129 

9 

45 

129 

9 

45 

Approved  for  Unlimited  Distribution 
Unclassified 
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Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score  (Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt.  x 
Grade) 

Result 

Grade 

Score  (Wt.  x 
Grade) 

Overall 

Score  % 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

None 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Less  than  1  sq.  ft.  refrigeration  area  required  per  assay. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

Assay  tube  kits  must  be  stored  at  4°C. 

Open  Architecture 

Info  Only 

Multiple  sources  of  consumables  or  self-designed 
assays 

Reagent  sources 
(List) 

Informational  only 

No 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Assays  are  self  contained.  No  waste  stream  other  than  assay  tube. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$25,000.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$25,000.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

Market  value  not  set.  Estimated  cost  per  assay  is  $65. 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

To  be  determined. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

System  saves  files  as  .fit,  .asy,  .xml,  .lddg,  and  .rst  extensions.  All  files  are  readable  with 
standard  text  editor,  excel,  or  html  display.  System  shipped  with  data  recovery  thumb 
drive. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external  ?) 

Informational  only 

No.  Assays  are  self  contained.  Recommend  decontaminating  sample  inlet. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  7. 

GRAND  TOTAL 

58 
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T-Cor  4  (1  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score 
(Wt.  x 
Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Singleplex  Target 
Identification 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

No  data 

100 

8 

80 

30 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

No  data 

250 

7 

70 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

100 

No  data 

No  assay 

0 

0 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

100 

No  data 

No  assay 

0 

0 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

No  data 

No  assay 

0 

0 

Multiplex  Target 
Identification 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

No  data 

N/A 

0 

0 

0 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

No  data 

N/A 

0 

0 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

50 

No  data 

N/A 

0 

0 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

50 

No  data 

N/A 

0 

0 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

No  data 

N/A 

0 

0 

Multiplex 

Capability 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 
run 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

1 

0 

0 

1 

0 

0 

2 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+ 
for  3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

Assay  Flexibility 

Number  of  Assay 
Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 

2  sources 

Score  =8+ 

for  3+ 

sources 

10 

80 

3+ 

10 

80 

3+ 

10 

80 

100 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Self 

10 

80 

Self 

10 

80 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one 
analyzer 

#  of  samples 

1=0  score; 
2-9 

10-50 

50+ 

10 

90 

4 

2 

18 

4 

2 

18 

20 

Run  Time 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

41 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready 
to  be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

30 

6 

48 

30 

6 

48 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

45 

1 

9 

45 

1 

9 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

Yes.  Qiagen  mini  kit  used  with  each  sample  before  analyzing. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

About  70  minutes  including  sample  preparation,  running  the  assay,  and  data 
interpretation. 

.a 

CZ5 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

6.2 

9 

90 

6.2 

9 

90 

95 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

<1 

10 

90 

<1 

10 

90 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

1  sq.  ft. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

1  lb. 

Power 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

96 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

120 

9 

45 

120 

9 

45 

Approved  for  Unlimited  Distribution 
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T-Cor  4  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score 
(Wt.  x 
Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

TCOR  Centrifuge 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Each  kit  required  <1  sq.  ft.  storage  space. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

Room  temperature 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
ml.  per  test 

Informational  only 

None 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$16,000.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$16,000.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$12.00 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  Tetracore. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Score  =  1  for  1  target 

Runs  only  saved  if  used  with  a  PC.  TCOR  uses  proprietary  software  not 
reviewed  in  the  evaluation. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

External  only. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  7. 

GRAND  TOTAL 

44 

Approved  for  Unlimited  Distribution  B-ll 
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NIDS  (1  of  2) 

Attribute 
Weight 
(1  to  10) 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Category 

Attribute 

Description 

Unit  of 

Measure 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 

(Wt.x 

Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 

(Wt.x 

Grade) 

Overall 

Score  % 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

1.00E+06 

4 

40 

10,000,000 

3 

30 

OJD  c 

b  ° 

S  ts 

11 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

2.50E+05 

5 

50 

2,500,000 

4 

40 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

1.00E+06 

2 

20 

>100,000,000 

1 

10 

30 

■So  -§ 

.S  1-1 

33 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

1.00E+08 

2 

20 

mmmm 

1 

10 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

50ng/mL 

6 

60 

50  ng/mL 

6 

60 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

1.00E+06 

4 

20 

10,000,000 

3 

15 

WD  C 

c3  -2 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

2.50E+05 

5 

25 

2,500,000 

4 

20 

2  « 
ftp 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

1.00E+06 

2 

10 

>100,000,00 

0 

1 

5 

26 

Z  2 

S 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

1.00E+08 

2 

10 

########## 

1 

5 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

50  ng/mL 

5 

25 

500  ng/mL 

4 

20 

x  >• 

"ft  2 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

run 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

5 

3 

27 

5 

3 

27 

s  o. 
s  (J 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

26 

Z 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

1 

1 

8 

1 

1 

8 

E 

« 

-< 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Company 

5 

40 

Company 

5 

40 

30 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1=0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

I 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

0 

10 

80 

0 

10 

80 

86 

H 

= 

P 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

16 

7 

63 

16 

7 

63 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

No 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

About  16  minutes  including  time  to  read  assay  strip. 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

1.6 

10 

100 

1.6 

10 

100 

100 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

0.08 

10 

90 

0.08 

10 

90 

z 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

No  ancillary  equipment  required. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

None  required. 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

100 

o 

CL 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

5 

10 

50 

5 

10 

50 

Approved  for  Unlimited  Distribution 
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NIDS  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

Approximately  12  hours/year  for  cleaning  the  SARIII  detector  glass  1-2  times/month. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

None 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Bag  of  50  assays  takes  up  approximately  1  sq.  ft. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

Room  temperature 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Assay  coupon  disposable.  No  liquid  waste  as  all  liquid  is  absorbed  into  coupon. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$6,500.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$6,500.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$45  per  5-plex  assay 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

None  expected;  SARIII  service  done  by  manufacturer  as  necessary. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

Data  files  can  be  downloaded  to  a  computer  and  stored  in  a  proprietary  database. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

None.  Assays  are  self-contained.  Recommend  decontaminating  reader  surface.  Bottom 
of  unit  is  removable  for  decontamination  of  assay  insertion  area  and  reader  lens. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  7. 

GRAND  TOTAL 

48 

Approved  for  Unlimited  Distribution  B-13 

Unclassified 


MagPix  (1  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score  (Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt.  x 
Grade) 

Result 

Grade 

Score  (Wt.  x 
Grade) 

Singleplex  Target 
Identification 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

none 

N/A 

N/A 

100,000 

6 

60 

40 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

none 

N/A 

N/A 

100,000 

6 

60 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

none 

N/A 

N/A 

10,000,000 

3 

30 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

none 

N/A 

N/A 

100,000,000 

2 

20 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

none 

N/A 

N/A 

1,000 

3 

30 

Multiplex  Target 
Identification 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

none 

N/A 

N/A 

10,000,000 

3 

15 

24 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

none 

N/A 

N/A 

1,000 

4 

20 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

none 

N/A 

N/A 

Could  not  test 

0 

0 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

none 

N/A 

N/A 

1,000,000,000 

2 

10 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

none 

N/A 

N/A 

1,000 

3 

15 

Multiplex 

Capability 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

run 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

50 

10 

90 

50 

10 

90 

84 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

none 

N/A 

N/A 

1 

1 

2 

Assay  Flexibility 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

3+ 

10 

80 

3+ 

10 

80 

100 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Self 

10 

80 

Self 

10 

80 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1  =  0  score; 
2-9 

10-50 

50+ 

10 

90 

96 

10 

90 

96 

10 

90 

100 

Run  Time 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

15 

3 

6 

15 

3 

6 

3 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

115 

0 

0 

115 

0 

0 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

115 

0 

0 

115 

0 

0 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

There  are  multiple  manual  sample  and  reagent  manipulation  steps. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

Raw  data  must  be  imported  into  Excel  and  interpreted.  Software  has  limited  functionality  to 
automatically  score  qualitative  results.  Approximately  15  required. 

05 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

39 

0 

0 

39 

0 

0 

38 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

1 

8 

72 

1 

8 

72 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

Approximately  6  sq.  ft. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

20  lbs. 

Power 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

No 

0 

0 

No 

0 

0 

31 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

240 

8 

40 

240 

8 

40 

Approved  for  Unlimited  Distribution 
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MagPix  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score  (Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt.  x 
Grade) 

Result 

Grade 

Score  (Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

13  hours  for  weekly  maintenance  run  of  approximately  15  minutes. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

Magnetic  bead  separator,  computer,  monitor,  plate  shaker,  multi-channel  pipets 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Less  than  1  sq.  ft.  refrigeration  area  required. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

Most  consumables  must  be  stored  at  4°C. 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Approximately  400uL  per  sample  or  4mL  per  full  96-well  plate. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$24,000.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$24,000.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$2-20  varies  with  batch  size  and  antibody  used. 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  Luminex;  12-Month  Preventative  Maintenance  kit  is  $150. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

Tab  delimited 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

Yes.  Internal  components  exposed  to  test  agent. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  7. 

GRAND  TOTAL 

44 

Approved  for  Unlimited  Distribution  B-15 
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Cartridge  Reader  (1  of  2) 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Category 

Attribute 

Weight 
(1  to  10) 

Description 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 

(Wt.x 

Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 

(Wt.x 

Grade) 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfii/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

6.68E+08 

2 

20 

100,000 

6 

60 

OJD  c 

b  ° 

S  ts 

11 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

3.01E+09 

1 

10 

100,000 

6 

60 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

1.50E+08 

2 

20 

10,000,000 

3 

30 

42 

"Sd  -a 

.5  1-1 

X 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

1.10E+10 

0 

0 

100,000,000 

2 

20 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

70  pg/mL 

9 

90 

100  ng/mL 

4 

40 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

6.68E+08 

2 

10 

100,000 

6 

30 

OX)  C 

c3  -2 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

3.01E+09 

1 

5 

100,000 

6 

30 

2  « 
ftp 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

1.50E+08 

2 

10 

100,000,000 

2 

10 

38 

=  2 

S 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

1.10E+10 

0 

0 

100,000,000 

2 

10 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

70  pg/mL 

9 

45 

10000  ng/mL 

3 

15 

* 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

run 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

5 

2 

18 

12 

5 

45 

s  o. 
s  (J 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

43 

s 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

1 

1 

8 

1 

1 

8 

E 

-< 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Company 

5 

40 

Company 

5 

40 

30 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1=0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

g 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

0 

10 

80 

0 

10 

80 

81 

H 

s 

p 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

30 

6 

54 

30 

6 

54 

c* 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

No 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

About  30  minutes. 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

13 

5 

50 

13 

5 

50 

69 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

0.68 

9 

81 

0.68 

9 

81 

Size 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

No  ancillary  equipment  required. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

No  ancillary  equipment  required. 

£ 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

No 

0 

0 

No 

0 

0 

38 

o 

Oh 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

18 

10 

50 

18 

10 

50 

Approved  for  Unlimited  Distribution 
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Cartridge  Reader  (1  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall 

Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

Maintenance  performed  by  MSD. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

None 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Informational  only 

Assays  shipped  in  individual  foil  pouches.  Each  assay  is  <1  sq.  ft. 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

Assays  must  be  refrigerated. 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Liquid  is  contained  within  the  assay  cartridge. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$85000  in  current  configuration. 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$85000  in  current  configuration. 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

To  be  determined. 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  done  by  MSD  as  necessary. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

Data  is  stored  on  device  and  can  be  downloaded  to  an  SD  card  in  .cvs  format.  Initially,  data 
had  to  be  transferred  to  an  excel  spreadsheet  for  analysis,  but  updated  firmware  displays 
results  summary  on  screen. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

No.  Assays  are  self-contained. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  6. 

GRAND  TOTAL 

45 
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Raptor  (1  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 

(Wt.x 

Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 

(Wt.x 

Grade) 

Singleplex  Target 
Identification 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfii/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

5.00E+04 

6 

60 

>5,000,000 

1 

10 

14 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

No  Claim 

0 

0 

50,000,000 

3 

30 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

100 

1.00E+05 

6 

60 

>100000000 

1 

10 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

100 

N/A 

0 

0 

N/A 

0 

0 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

lOng/mL 

7 

70 

^10,000  ng/ml 

2 

20 

Multiplex  Target 
Identification 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

1.00E+06 

4 

20 

>5,000,000 

1 

5 

8 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

2.50E+05 

5 

25 

50,000,000 

3 

15 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

1.00E+06 

4 

20 

N/A 

0 

0 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

N/A 

0 

0 

N/A 

0 

0 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

50  ng/mL 

6 

30 

>ug/mL 

0 

0 

Multiplex 

Capability 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

run 

1=0  score;  2- 
7 

8-19 

20-34+ 

10 

90 

4 

2 

18 

4 

2 

18 

18 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for  1 
target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

4 

4 

8 

1 

1 

2 

Assay  Flexibility 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for  1 

source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

1 

1 

8 

1 

1 

8 

30 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Company 

5 

40 

Company 

5 

40 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer. 

#  of  samples 

1  =  0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Run  Time 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

76 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

0 

10 

80 

0 

10 

80 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

15 

7 

63 

28 

5 

45 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

No 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

Approximately  28  minutes,  includes  time  required  for  baseline  correction  (required  for  a 
new  assay  coupon). 

Size 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

12.3 

5 

50 

12.3 

10 

100 

91 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

1 

8 

72 

1 

8 

72 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

square  feet 

Informational  only 

Less  than  0.5  sq.  ft. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

About  1  lb. 

Power 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

100 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

30 

10 

50 

30 

10 

50 

Approved  for  Unlimited  Distribution 
Unclassified 
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Raptor  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

26  hours/year  for  weekly  maintenance  run. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

A/C  Adapter,  battery  charger 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Each  kit  measures  2.5  x  3  inches  (0.05  sq.  ft.). 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

Refrigeration 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Approximately  lOmL  per  test. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$49,500.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$49,500.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$150  per  custom  cartridge  (re-usable). 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  Research  International. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Informational  only 

Proprietary  software  required  to  download  files. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

Yes.  Internal  system  components  are  exposed  to  test  agent. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  6. 

GRAND  TOTAL 

37 

Approved  for  Unlimited  Distribution  B-19 

Unclassified 


SpinDx  (1  of  2) 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Category 

Attribute 

Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 

(Wt.x 

Grade) 

Claim 

Grade 

Score  (Wt. 
x  Grade) 

Result 

Grade 

Score 

(Wt.x 

Grade) 

Overall  Score  % 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfii/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

N/A 

0 

0 

0 

0 

0 

OJD  S3 

c3  2 

H  '■§ 

S  c 

11 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

N/A 

0 

0 

0 

0 

0 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

N/A 

0 

0 

0 

0 

0 

0 

"So  -§ 

a  hh 

33 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

100 

N/A 

0 

0 

0 

0 

0 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

N/A 

0 

0 

0 

0 

0 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

N/A 

0 

0 

0 

0 

0 

wd  a 
«  -2 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

N/A 

0 

0 

0 

0 

0 

3  «s 

.2-  c 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

N/A 

0 

0 

0 

0 

0 

0 

=  2 

S 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Vims: 

10e7-10e9 

Vims: 

1000-10e6 

Vims: 

1-100/mL 

10 

50 

N/A 

0 

0 

0 

0 

0 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

N/A 

0 

0 

0 

0 

0 

*  & 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 

run 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

20 

8 

72 

2 

1 

9 

—  a 

3  & 

s  (J 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+  for 
3+  targets 

10 

20 

1 

1 

2 

1 

1 

2 

10 

s 

Number  of  Assay  Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 
2  sources 

Score  =8+  for 
3+  sources 

10 

80 

1 

1 

8 

1 

1 

8 

E 

< 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =  10 
for  Self 
designed 

10 

80 

Company 

5 

40 

Company 

5 

40 

30 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one  analyzer 

#  of  samples 

1=0  score; 
2-9 

10-50 

50+ 

10 

90 

15 

4 

36 

15 

4 

36 

40 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

0 

10 

20 

0 

10 

20 

g 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready  to 
be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

20 

7 

56 

20 

7 

56 

78 

H 

= 

s 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

15 

8 

72 

15 

8 

72 

c* 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

Yes.  Sample  is  diluted  into  Sample  Dilution  solution  prior  to  loading  pouch. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

Result  automatically  displayed  on  LCD  screen. 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

4 

9 

90 

4 

9 

90 

95 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

0.2 

10 

90 

0.2 

10 

90 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

None  required. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

N/A 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

100 

o 

CL 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

-200 

8 

40 

25 

10 

50 

Approved  for  Unlimited  Distribution 
Unclassified 
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SpinDx  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of 

Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

_  Score  (Wt. 

Grade  x  Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

None 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Each  kit  measures  7.125  x  7.25  inches  (0.36  sq.  ft.). 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

None.  Room  temperature  storage. 

Open  Architecture 

Info  Only 

Multiple  sources  of  consumables  or  self-designed 
assays 

Reagent 
sources  (List) 

Informational  only 

No 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate 
mL  per  test 

Informational  only 

Approximately  lmL  liquid  remains  in  syringes  used  for  loading  assay.  Assay  pouch  is 
self-contained.  Sample  dilution  vial  contains  approximately  5mL  liquid. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$38,500.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$38,500.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$200.00 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  Sandia  National  Laboratory  staff. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Score  =  1  for  1  target 

Results  viewed  on  screen.  Results  can  be  downloaded  using  a  proprietary  software  and 
laptop. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

External  decontamination  recommended. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  4. 

GRAND  TOTAL 

34 

Approved  for  Unlimited  Distribution  B-21 

Unclassified 


Spirit  (1  of  2) 

Attribute 
Weight 
(1  to  10) 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Category 

Attribute 

Description 

Unit  of  Measure 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score 
(Wt.  x 
Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Overall  Score  % 

Singleplex  Sensitivity  - 
Gram+ 

10 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

1000 

7 

70 

13000 

5 

50 

m  a 
«  .2 

Singleplex  Sensitivity  - 
Gram- 

10 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

100 

100 

8 

80 

1000 

7 

70 

E-1  « 

3  c 

a  '-g 

Singleplex  Sensitivity  - 
DNA  Virus 

10 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

100 

N/A 

0 

0 

No  assay 

0 

0 

24 

M  -e 

Singleplex  Sensitivity  - 
RNA  Virus 

10 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/rnl 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

100 

1000 

0 

0 

No  assay 

0 

0 

Singleplex  Sensitivity  - 
Toxin 

10 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

100 

N/A 

0 

0 

No  assay 

0 

0 

Multiplex  Sensitivity  - 
Gram+ 

5 

Limit  of  detection  (Gram  +  bacteria),  B.  anthracis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

1000 

7 

35 

26000 

5 

25 

M  E 

13  .2 

Multiplex  Sensitivity  - 
Gram- 

5 

Limit  of  detection  (Gram  -  bacteria),  Y.  pestis 

cfu/ml 

Bacteria: 

10e7-10e9 

Bacteria: 

1000-10e6 

Bacteria: 

1-100 

10 

50 

100 

8 

40 

2600 

7 

35 

3  s 
%  ’■§ 

=  2 

S 

Multiplex  Sensitivity  - 
DNA  Virus 

5 

Limit  of  detection  (DNA  Virus),  Vaccinia 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

50 

N/A 

0 

0 

No  assay 

0 

0 

24 

Multiplex  Sensitivity  - 
RNA  Virus 

5 

Limit  of  detection  (RNA  Virus),  VEE 

pfu/ml 

Virus: 

10e7-10e9 

Virus: 

1000-10e6 

Virus: 

1-100/mL 

10 

50 

N/A 

0 

0 

No  assay 

0 

0 

Multiplex  Sensitivity  - 
Toxin 

5 

Limit  of  detection  (Toxin),  Botulinum  Type  A  Toxin 

mass/mL 

Toxins: 

ug/mL 

Toxins: 

ng/mL 

Toxins: 

pg/mL 

10 

50 

N/A 

0 

0 

No  assay 

0 

0 

JS.§ 

Multiplex 

9 

Number  of  reportable  agents  per  run 

#  of  agents  per 
run 

1=0  score; 
2-7 

8-19 

20-34+ 

10 

90 

10 

5 

45 

2 

1 

9 

s  §■ 

Multiplex 

2 

Number  of  individual  targets  per  test 

#  individual 
targets  per  test 

Score  =  1  for 

1  target 

Score  =  5  for 
2  targets 

Score  =8+ 
for  3+  targets 

10 

20 

10 

9 

18 

1 

1 

2 

10 

2 

Number  of  Assay 
Sources 

8 

Multiple  sources  of  consumables  or  self-designed 
assays 

#  of  sources 

Score  =  1  for 

1  source 

Score  =  5  for 

2  sources 

Score  =8+ 

for  3+ 

sources 

10 

80 

3+ 

10 

80 

1 

1 

8 

E 

« 

Assay  Development 

8 

New  assay  integration 

Ease  of 
acquisition 

Score  =  1  for 
Financially 
limiting 

Score  =  5  for 
Company 
designed 

Score  =10 
for  Self 
designed 

10 

80 

Company 

5 

40 

Company 

5 

40 

30 

Batch 

Size 

Number  of  Samples 

9 

If  looking  for  a  single  agent,  the  number  of  samples 
that  can  be  processed  at  the  same  time  by  one 
analyzer 

#  of  samples 

1=0  score; 
2-9 

10-50 

50+ 

10 

90 

1 

0 

0 

1 

0 

0 

0 

Boot  Up 

2 

Time  required  for  warm-up  and  calibration  prior  to 
sample  analysis 

Minutes 

14+ 

6<14 

0<6 

10 

20 

5 

9 

18 

5 

9 

18 

i 

Sample  Preparation  Time 

8 

Time  required  from  receipt  of  sample  until  it  is  ready 
to  be  analyzed 

Minutes 

35+ 

15<35 

0<15 

10 

80 

5 

9 

72 

5 

9 

72 

71 

H 

Analysis  Time 

9 

Time  required  to  provide  a  test  answer  after  the 
analysis  has  been  initiated 

Minutes 

35+ 

15<35 

0<15 

10 

90 

30 

5 

45 

30 

5 

45 

Pi 

Sample  Preparation 

Info  Only 

Sample  preparation  required? 

Yes/No 

Informational  only 

Yes.  Sample  is  diluted  into  Sample  Dilution  solution  prior  to  loading  pouch. 

Total  Analysis  Time 

Info  Only 

The  duration  of  time  to  provide  final  identification  of 
target  from  raw  data 

Minutes 

Informational  only 

Result  automatically  displayed  on  LCD  screen. 

Weight 

10 

Approximate  weight  of  instrument  with  power  source 

Pounds 

24-36 

8<24 

2<8 

10 

100 

11 

7 

70 

11 

7 

70 

84 

Instrument  Size 

9 

Approximate  footprint  of  instrument  only  in 
operational  mode 

Square  feet 

3.5+  sq.  ft. 

1.5<3.5  sq. 
ft. 

0  <1.5  sq.  ft. 

10 

90 

0.31 

10 

90 

0.31 

10 

90 

.a 

C/3 

Logistical  Footprint 

Info  Only 

Approximate  footprint  of  instrument  with  ancillary 
equipment 

Square  feet 

Informational  only 

None  required. 

Ancillary  Equipment 
Weight 

Info  Only 

Approximate  weight  of  ancillary  equipment 
(microcentrifuge,  plate  shaker,  plate  washer,  laptop) 

Pounds 

Informational  only 

N/a 

fe 

Power  Requirements 

8 

Battery  supplied 

Yes/No 

No  =  0;  Yes  =  10 

10 

80 

Yes 

10 

80 

Yes 

10 

80 

100 

O 

CL 

Power  Requirements 

5 

Expected  power  required 

Watts 

700+ 

301-700 

0-300 

10 

50 

45 

10 

50 

45 

10 

50 

Approved  for  Unlimited  Distribution 
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Spirit  (2  of  2) 

Category 

Attribute 

Attribute 
Weight 
(1  to  10) 

Description 

Unit  of  Measure 

Grading  Scale 

Ideal  Instrument 

Vendor  Supplied  Information 

Hardware  Testing 

Overall  Score  % 

Low 

(1-3) 

Medium 

(4-7) 

High 

(8-10) 

Grade 

Score 
(Wt.  x 
Grade) 

Claim 

Grade 

Score 
(Wt.  x 
Grade) 

Result 

Grade 

Score 
(Wt.  x 
Grade) 

Logistical  Support 

User  Maintenance 

Info  Only 

Time  required  for  Preventative  Maintenance 

Hours  per  year 

Informational  only 

All  maintenance  is  performed  by  the  company. 

Ancillary  Equipment 

Info  Only 

List  ancillary  equipment  requirement  for  operation  of 
this  instrument 

List  ancillary 
equipment 

Informational  only 

None 

Consumables 

Info  Only 

Storage  area  required  for  consumables 

Square  feet 

Informational  only 

Each  kit  measures  7.125  x  7.25  inches  (0.36  sq.  ft.) 

Consumables 

Requirements 

Info  Only 

Special  conditions  required  for  consumables  (e.g., 
refrigeration,  high  humidity) 

Conditions 

Informational  only 

None.  Room  temperature  storage. 

Waste  Management 

Info  Only 

Approximate  volume  of  waste  streams 

Approximate  mL 
per  test 

Informational  only 

Approximately  lmL  liquid  remains  in  syringes  used  for  loading  assay. 
Assay  pouch  is  self-contained.  Sample  dilution  vial  contains  approximately 
5mL  liquid. 

Costs 

Quoted  Cost 

Info  Only 

System  cost  as  quoted  or  listed 

$ 

Informational  only 

$38,500.00 

Purchase  Cost 

Info  Only 

Actual  system  cost  as  purchased 

$ 

Informational  only 

$38,500.00 

Reagent  Cost 

Info  Only 

Reagent  cost  per  sample 

$ 

Informational  only 

$200  per  sample  chip  (re-usable). 

Service  Cost 

Info  Only 

Cost  for  yearly  service 

$ 

Informational  only 

Service  as  necessary  by  Seattle  Sensors  Systems. 

Compatibility/ 

Interchangeability 

Interoperability 

Info  Only 

Types  of  data  files  for  export  (e.g.,  ASCII,  XML,  tab 
delimited  text) 

File  types  (list) 

Score  =  1  for  1  target 

Results  viewed  on  screen.  Results  can  be  downloaded  using  a  proprietary 
software  and  laptop. 

Usability 

Decontamination 

Requirements 

Info  Only 

Is  decontamination  required? 

Yes/No  (If  Yes, 
internal  and/or 
external?) 

Informational  only 

External  decontamination  recommended. 

Maturity 

Readiness  Level 

Info  Only 

Current  Technical  Readiness  Level  (TRL) 

TRL 

Informational  only 

Based  on  subject  matter  expert  feedback,  the  TRL  is  5. 

GRAND  TOTAL 

41 

Approved  for  Unlimited  Distribution  B-23 
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GENERAL  INFORMATION 

Instrument  Name 

FilmArray 

Instrument  Serial  Number 

FA2070 

Firmware  Version 

Master:  1.3.4,  Thermocycler:  1.3.27,  Valve:  1.6.9,  Protocol:  2 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

40  (laptop,  barcode  reader,  device,  and  pouch  preparation 
assembly) 

System  Footprint  (w”  x  d”  x  h”) 

10  x  15.5x6.5 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

34  x  20  x  11  (including  laptop,  barcode  reader,  device,  pouch 
loading  station,  cables,  and  mouse) 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

No 

Comments:  Manual  is  well  written,  easy  to  understand.  Manual  also  has  troubleshooting  section  and  safety  information. 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

No 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

System  is  shipped  with  laptop  and  barcode  reader.  Barcode 
reader  is  optional. 

Were  there  any  special  power  requirements? 

No.  11  OVA/C 

How  much  time  was  required  to  set  up  the  instrument? 

15  minutes 

How  long  did  training  take? 

45  minutes  to  read  through  manual 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

Yes 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

N/A 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

Barcode  reader  didn’t  function  during  pre-assessment.  Required 
calibration  using  barcodes  included  in  User  Manual.  Calibration 
procedure  took  approximately  30  seconds. 

Comments:  None 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

BioThreat  Panel  v2.4 

Room  Temp 

6  months 

Are  there  any  reagents  that  require 
additional  preparation  by  the  end- 
user? 

BioThreat  Panel  Kit  contains  all  reagents  required  except  dilution  buffer.  Kit 
contains  array,  array  hydration  buffer,  sample  buffer,  syringes  with  canulas,  and 
sample  transfer  pipette. 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

Dilution  buffer  if  samples  will  be  diluted  prior  to  assessment. 

Approved  for  Unlimited  Distribution 
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GENERAL  INFORMATION 

Instrument  Name 

RAZOR 

Instrument  Serial  Number 

EX4353 

Firmware  Version 

rev06Jan05  2012 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

11 

System  Footprint  (w”  x  d”  x  h”) 

9.25x4.5x8 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

N/A 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

No 

Comments:  None 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

No 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

N/A 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

<  5  minutes 

How  long  did  training  take? 

45  minutes  to  read  instruction  manual  and  watch  videos 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

No 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

N/A 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

No 

Comments:  None 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

Reagent  Grade  Water 

18-25  C  and  85%  Humidity 

N/A 

Unknown  Sample  Bottle 

18-25  C  and  85%  Humidity 

N/A 

The  10  Pouch  (PATH-ASY-0061) 

18-25  C  and  85%  Humidity 

6  months 

Are  there  any  reagents  that  require 
additional  preparation  by  end-user? 

No 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

No 

Approved  for  Unlimited  Distribution 
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GENERAL  INFORMATION 

Instrument  Name 

Genedrive 

Instrument  Serial  Number 

9fl3f7f-95.6-647-i 

Firmware  Version 

Unknown 

Is  this  a  COTS  Instrument? 

Yes — but  is  still  in  development 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

1.2 

System  Footprint  (w”  x  d”  x  h”) 

5  x  7.5x5 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

Requires  up  to  an  additional  100  in2  workspace  for  assay  set  up 
and  storage  of  ancillary  equipment 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Battery  packs  were  not  delivered  with  units. 

Was  the  training/operator  manual  included? 

Yes 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

A/C  adapter  or  battery  pack 

Comments:  There  were  difficulties  in  shipment  of  supplies  from  United  Kingdom  to  Edgewood  Chemical  Biological  Center. 
For  example,  batteries  were  held  in  customs  causing  a  delay  in  delivery. 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

No.  Everything  is  stored  at  room  temperature. 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

2B  BlackBio  BlackLight  Card,  Assay  cartridges  with  lyophilized 
primer/probes,  Ready-To-Go  PureTaq  PCR  Beads  (GE),  Biopsy 
punch,  PCR  Grade  water,  cartridge  cap  assemblies.  As  tested  for 
assessment,  cartridges  were  empty  and  primer/probes  delivered 
separately  as  lyophilized  products. 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

Less  than  5  minutes.  Assay  set  up  required  about  15  minutes. 

How  long  did  training  take? 

About  a  20  minute  demonstration. 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

No.  Company  had  demonstrated  use  of  the  device  and 
accessories. 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

Worked  nearly  exclusively  with  Director  of  Diagnostics. 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

None 

Comments:  We  requested  engineering  software  from  Epistem  that  allowed  us  to  view  data  on  the  device.  The  software  required 
a  laptop.  This  would  not  be  required  for  field  use  or  laboratory  assessment  of  full  production  model. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

Assay  Cartridges  with  Lyophilized 
Primer/Probes 

Room  Temperature 

1  year 

2B  BlackBio  BlackLight  Paper 

Room  Temperature 

Unknown 

Ready-To-Go  PureTaq  PCR  Beads 

Room  Temperature 

Approximately  2  years 

Separated  Lyophilized  Primer/Probes 

Room  Temperature 

1  year 

PCR  Grade  Water 

Room  Temperature 

1  year 

Are  there  any  reagents  that  require 
additional  preparation  by  the  end-user? 

When  testing  with  separated  lyophilized  primer/probes,  primer/probes  were 
reconstituted  with  PCR  grade  water,  stored  at  4°C  (short  term)/  20°C  (long  term) 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

DNA  purification  kit  for  evaluation  of  system  with  purified  DNA 

C-4 
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GENERAL  INFORMATION 

Instrument  Name 

LIAT 

Instrument  Serial  Number 

Ml-D-00065 

Firmware  Version 

1.4.0 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

8.3 

System  Footprint  (w”  x  d”  x  h”) 

4.5  x  10x7.5 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

N/A 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

No 

Comments:  N/A 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

Kept  at  4°C 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

External  battery 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

10  minutes 

How  long  did  training  take? 

30  minutes  to  read  instruction  manual. 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

No 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

No 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

No 

Comments:  Product  should  be  operated  between  15°C  and  32°C.  Analyzer  should  be  level,  free  of  vibrations,  and  out  of  direct 
sunlight. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

Assay  Tubes 

4°C 

1  year 

Are  there  any  reagents  that  require 
additional  preparation  by  the  end- 
user? 

No 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

No 

Approved  for  Unlimited  Distribution 
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GENERAL  INFORMATION 

Instrument  Name 

T-Cor  4 

Instrument  Serial  Number 

XXX 

Firmware  Version 

XXX 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

6.2 

System  Footprint  (w”  x  d”  x  h”) 

9x7.5  x  2.5 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

N/A 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

No 

Comments:  None 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

No 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

Mini  centrifuge 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

<  5  minutes 

How  long  did  training  take? 

30  minutes  to  read  instruction  manual. 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

No 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

N/A 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

No 

Comments:  None 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

Rehydration  Buffer 

15-30°C 

1  year 

Test  Sample  Tubes  in  bag 

15-30°C 

1  year 

Positive  Control  Tubes 

15-30°C 

1  year 

Are  there  any  reagents  that  require 
additional  preparation  by  end-user? 

No 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

No 

Approved  for  Unlimited  Distribution 
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GENERAL  INFORMATION 

Instrument  Name 

NIDS 

Instrument  Serial  Number 

20110047 

Firmware  Version 

V2.0.3.29 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

~  3.5 

System  Footprint  (w”  x  d”  x  h”) 

2.5x4.75x4 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

16  x  9.5  x  4 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Instruction  manual  included. 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

Device  has  USB  rechargeable  battery  pack  that  requires  a 
computer  or  other  USB  power  source. 

Comments:  None 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

No 

Were  the  reagents  received  as  required  by  the 
manufacturer 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

Laptop  computer  for  device  power  and  download  of  results. 

Were  there  any  special  power  requirements? 

USB 

How  much  time  was  required  to  set  up  the  instrument? 

<5  minutes  excluding  installation  of  optional  computer  software. 

How  long  did  training  take? 

Simple  to  learn.  <10  minutes  including  sample  preparation. 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

No 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

N/A 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

No 

Comments:  Device  appears  to  discharge  quickly  when  not  connected  to  computer  or  computer  in  standby  mode. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

5-Plex  1  Cartridge 

Room  Temp 

FAQ  lists  2  years  from  date  of 
manufacture  (DOM).  ANP  says  1  year 
from  receipt. 

5-Plex  2  Cartridge 

Room  Temp 

FAQ  lists  2  years  from  DOM.  ANP  says 

1  year  from  receipt 

Are  there  any  reagents  that  require 
additional  preparation  by  end-user? 

PBSTK,  PBS,  or  other  dilution  buffers. 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

PSBTK  is  available  from  company  but  not  originally  supplied. 

Approved  for  Unlimited  Distribution 
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GENERAL  INFORMATION 

Instrument  Name 

MagPix 

Instrument  Serial  Number 

MagPx 122 14702 

Firmware  Version 

1.1.539 

Is  this  a  COTS  Instrument? 

4.2  Build  1324 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

38.5 

System  Footprint  (w”  x  d”  x  h”) 

6.5  x  23x  16.5 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

26  x  23  x  16.5 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes.  Manual  included  on  disc.  Training  videos  included  with 
software. 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

A  computer  is  required  to  control  the  system. 

Comments:  System  was  set  up  by  Luminex  personnel. 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

Refrigeration 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

Vortemp  for  plate  incubations,  vortexer,  magnetic  96-well  format 
base  for  bead  trapping,  magnetic  bead  trapper  for  1.5mL 
microcentrifuge  tubes,  multi-channel  micropipette,  sonicator  for 
preventive  maintenance 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

90  minutes 

How  long  did  training  take? 

240  minutes 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

Yes.  Discussed  selection  of  bead  regions  and  overall  assay 
design. 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

Dealt  directly  with  Sr.  Field  Application  Specialist  and/or 
Director  of  Government  Business  Development. 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

System  requires  an  initial  calibration  and  performance 
verification  which  must  be  repeated  weekly. 

Comments:  Suggested  preventive  maintenance  is  extensive  including  weekly,  monthly,  bi-annual,  and  annual  operations. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

PBS  +  1  %  BSA  (“Block”) 

4°C 

6  months  -  1  year 

PBS  +  0.1  %  Tween-20  (“Wash  Buffer”) 

4°C 

6  months  -  1  year 

MagPlex  Beads 

4°C  in  dark 

6  months  -  1  year 

Antibody  pairs  as  required  for  assay 

Varies,  most  likely  either  +4  or  -20°C 

Varies 

EZ-Link  Micro  Sulfo-NHS-LC- 

Biotinylation  Kit  (Pierce  Cat  #21935) 

Varies  by  component 

Varies 

Biotin  Quantitation  Kit  (Pierce  Cat 
#28005) 

Varies  by  component 

Varies 

Antibody  Coupling  Kit  (Luminex  Cat 
#B29045) 

4°C 

6  months  -  1  year.  One  component  is 
single  use. 

Streptavidin,  R-phycoerythin  (Invitrogen 
Cat  #S866) 

4°C  in  dark 

6  months  -  1  year 
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Are  there  any  reagents  that  require 
additional  preparation  by  the  end- 
user? 

See  section  above.  System  is  open  architecture  and  requires  end-user  to  develop 
assays  including  labeling  detector  antibodies  with  biotin  and  coupling  capture 
antibodies  to  MagPlex  beads. 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

The  antibody  coupling  kit  is  available  separately  from  Luminex  and  is  not  included 
with  device.  All  other  reagents  listed  above  are  available  from  other  sources. 
Luminex  supplies  a  list  of  select  vendors. 
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GENERAL  INFORMATION 

Instrument  Name 

Cartridge  Reader 

Instrument  Serial  Number 

0.0.4.2004 

Firmware  Version 

A:  410091015113 

B:  410091015114 

Is  this  a  COTS  Instrument? 

No 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

~  3.5 

System  Footprint  (w”  x  d”  x  h”) 

7  x  13.5  x  8 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

No  ancillary  equipment  required  after  firmware  upgrade.  Prior  to 
upgrade,  a  laptop  was  required  to  analyze  raw  data. 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Instruction  manual  included 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

Power  cord  supplied  with  device 

Comments:  None. 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

Refrigeration,  wet  ice 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

None  after  firmware  upgrade,  laptop  before  upgrade 

Were  there  any  special  power  requirements? 

1 10V  A/C,  no  battery 

How  much  time  was  required  to  set  up  the  instrument? 

No  set-up  required  other  than  plugging  in  power  cord 

How  long  did  training  take? 

Simple  to  learn  to  operate.  <10  minutes 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

Yes.  Inquired  whether  data  existed  for  levels  of  detection 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

No.  Worked  mainly  with  CEO  of  company 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

Yes 

Comments:  Training  initially  took  longer  because  user  was  required  to  understand  how  to  transfer  data  to  an  SD  card  and  import 
it  into  Excel  for  analysis  using  a  customized  spreadsheet  from  MSD.  Upon  request,  MSD  installed  an  updated  firmware  that 
removes  the  requirement  of  exporting  the  raw  data  to  excel  for  analysis.  For  initial  calibration,  lot  parameters  had  to  be 
downloaded  to  the  device  from  a  memory  card  supplied  with  the  assays. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

ECBC  Assay  Panel  Cartridge 

4°C 

~  1  year 

Are  there  any  reagents  that  require 
additional  preparation  by  end-user? 

No 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

No 
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GENERAL  INFORMATION 

Instrument  Name 

RAPTOR 

Instrument  Serial  Number 

SF 10047 

Firmware  Version 

1.39 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

12.3  w/o  battery,  2.5  w/battery 

System  Footprint  (w”  x  d”  x  h”) 

7.3  x  6.8  x  10.8 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

N/A 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

No.  Device  was  shipped  with  battery  pack  and  A/C  adapter. 

Comments:  Shipped  incorrect  A/C  adapter.  Received  correct  adapter  to  power  device.  Research  International  indicated 
looseness  of  power  input  (female  end  on  device)  was  a  design  feature. 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

Reagents  require  storage  at  4°C. 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

None.  Computer  is  optional. 

Were  there  any  special  power  requirements? 

Requires  battery  pack  or  A/C. 

How  much  time  was  required  to  set  up  the  instrument? 

5  minutes 

How  long  did  training  take? 

About  240  minutes  to  read  manual  and  understand  system 
function. 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

Yes.  Contacted  Dr.  David  McCrae. 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

No.  Talked  directly  with  Vice  President. 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

Device  conducts  a  system  check  at  power  up.  Takes  seconds. 

Comments:  Waste  bag  vent  port  is  plumbed  to  the  sample  reservoir  which  could  cause  contamination/de  filing  of  sample. 
Manual  is  vague  and  contains  typographical  errors.  Battery  pack  is  non-rechargeable  although  a  rechargeable  option  is  available, 
battery  discharged  after  less  than  two  days  of  testing.  Color  coding/labeling  of  device  inconsistent  and  incorrect.  System 
contains  a  reagent  cooler  which  needs  to  be  frozen  at  -20°C  prior  to  use.  Maintains  reagents  at  temperature  <29°C  for  24  hours. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

RAPTOR  Bioassay  Coupon  Kit 

4°C 

Production  qualified  for  3  to  6  months  at 
20°C. 

Wash  Buffer 

4°C 

Assigned  1  year  expiration. 

Are  there  any  reagents  that  require 
additional  preparation  by  end-user? 

Yes.  Wash  buffer  is  8.3mM  Phosphate  Buffer,  pH  7.2  with  0.05%  Triton  X-100. 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

Yes.  Wash  buffer.  Vice  President  recommended  ordering  pre -weighed  packets  from 
Sigma. 
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GENERAL  INFORMATION 

Instrument  Name 

SpinDx 

Instrument  Serial  Number 

Unknown 

Firmware  Version 

Unknown 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

3.5 

System  Footprint  (w”  x  d”  x  h”) 

6x6x6 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

Dependent  on  laptop  used.  Communication  with  laptop  is 
wireless  via  Bluetooth;  therefore,  laptop  does  not  have  to  be 
located  adjacent  to  device. 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No.  System  shipped  in  a  pelican  style  case. 

Were  all  the  parts  included  in  the  shipment? 

Yes 

Was  the  training/operator  manual  included? 

Yes.  Shipment  included  software  and  training  video.  Training 
did  not  cover  data  analysis. 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

System  is  powered  by  a  built-in  rechargeable  battery  pack. 

Comments:  None 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

Discs  and  lyophilized  reagents  require  storage  at  4°C. 

Were  the  reagents  received  as  required  by  the 
manufacturer 

Yes.  They  were  received  on  ice  packs. 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

None 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

Installation  of  Bluetooth  adapter,  virtual  serial  connection,  and 
device  OS  software  took  about  30  minutes. 

How  long  did  training  take? 

10  minutes 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

Yes.  Asked  them  how  to  interpret  the  data.  No  information  on 
data  interpretation  was  sent  with  shipment. 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

No.  Spoke  directly  to  assay  developer. 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

Not  initially. 

Comments:  This  system  requires  pipetting  small  volumes  of  microbeads  that  may  get  trapped  in  a  lOuL  micropipette  tip.  A 
single  volume  micropipette  and  appropriate  larger  bore  tips  were  shipped  with  the  unit.  Unit  works  via  a  Bluetooth  enabled 
virtual  serial  port.  After  running  initial  assays,  called  the  developer  with  concerns  about  system  performance.  At  this  time,  they 
shipped  calibrator  beads  to  us.  Calibrator  beads  indicated  there  is/are  performance  issues  with  instrument  and  data  is  unreliable. 
Will  not  assess  detection  of  targets. 

REQUIRED  REAGENTS,  STORAGE,  AND  SHELF  LIFE 

Reagent 

Storage 

Shelf  Life 

Disc 

4°C 

“A  couple  of  months.” 

Lyophilized  reagents  for  each  assay 

4°C 

“A  couple  of  months.” 

Are  there  any  reagents  that  require 
additional  preparation  by  end-user? 

No 

Are  there  any  reagents  that  were  not 
supplied  by  the  company? 

No 
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GENERAL  INFORMATION 

Instrument  Name 

SPIRIT 

Instrument  Serial  Number 

Instrument  does  not  have  serial  number 

Firmware  Version 

Unknown 

Is  this  a  COTS  Instrument? 

Yes 

ACTUAL  EQUIPMENT  SPECS 

Weight  (lbs) 

3 

System  Footprint  (w”  x  d”  x  h”) 

11  x7x7.5 

Total  footprint  with  ancillary  equipment  (w”  x  d”  x  h”) 

N/A 

SHIPMENT  OF  EQUIPMENT 

Was  there  movement  during  shipment? 

No 

Were  all  the  parts  included  in  the  shipment? 

Multiple  versions  sent  via  email. 

Was  the  training/operator  manual  included? 

Received  via  email. 

Was  additional  equipment  required  to  power  on  the 
device?  If  so,  explain. 

Yes.  Unit  shipped  with  an  external,  rechargeable  battery  and  A/C 
power  adapter. 

Comments:  None 

SHIPMENT  OF  REAGENTS 

Did  reagents  require  any  special  shipping? 

Reagents  shipped  cold.  Subsequent  shipments  were  received 
without  refrigeration  and  box  did  not  indicate  contents  required 
refrigeration  upon  receipt. 

Were  the  reagents  received  as  required  by  the 
manufacturer? 

Yes 

SET  UP  OF  EQUIPMENT 

List  ancillary  equipment  required 

Laptop  computer,  Serial-USB  adapter,  blunt  end  syringes 

Were  there  any  special  power  requirements? 

No 

How  much  time  was  required  to  set  up  the  instrument? 

See  Comments. 

How  long  did  training  take? 

Two  hours  once  on  site. 

Did  you  call  the  company  with  questions  prior  to  starting 
your  first  run? 

Yes 

Did  the  company  have  a  dedicated  tech  support  person 
that  was  able  to  help  you? 

Yes 

Are  any  initial  calibrations  required?  If  so,  insert  a 
description  and  time  required  in  Comments. 

Yes 

Comments:  Set-up  involved  opening  a  pelican  case,  installing  two  buffers  and  a  waste  tube,  pre-wetting  of  the  sensors, 
installation  of  specific  and  reference  sensors,  priming  of  the  system,  and  referencing  the  sensors  to  a  sucrose  solution.  Normally 
this  process  should  take  no  more  30  minutes,  but  this  system  was  fraught  with  problems. 

The  first  system  arrived  without  a  functional  peristaltic  pump.  This  system  was  replaced  with  a  device  that  appeared  to  work  but 
did  not  record  data  from  the  Y.  pestis  chip  (bad  chip/bad  software/unknown  problem).  The  following  day,  the  system  became 
entirely  unresponsive  to  user  input.  The  company  offered  phone  technical  support  and  sent  various  versions  of  the  instrument 
operating  software  to  try  to  remedy  the  issues.  The  company  finally  replaced  this  device  with  the  refurbished  initial  unit.  The 
initial  device  again  arrived  without  a  functional  peristaltic  pump.  At  this  point,  we  requested  that  Seattle  Sensors  travel  to  ECBC 
with  a  functional  device.  They  set  up  and  tested  the  device  in  our  labs  with  our  laptop  and  gave  a  training/introduction  to  the 
system.  This  third  device  appears  to  function  appropriately. 

Initial  calibrations:  Sensor  chips  must  be  calibrated  against  a  high  refractive  index  solution  (sucrose)  before  first  use.  This 
involves  injecting  a  20-30%  sucrose  solution  into  the  sample  loop  and  allowing  it  to  cover  sensor  surfaces.  The  computer 
software  then  has  an  algorithm  to  reference  the  sensors  to  the  initiation  solution.  Additionally,  running  buffer  must  be  passed 
across  the  sensors  for  enough  time  prior  to  sample  analysis  to  allow  a  steady  baseline  to  develop.  The  time  required  for  this  is 
subject  to  variations  in  the  antibodies  coated  onto  the  sensor  surfaces  and  is  empirical. 

C-13 

Approved  for  Unlimited  Distribution 
Unclassified 


Appendix  D:  Operational  Assessment  Table 
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Instrument: 

Weather  Conditions: 

Time  of  Dav: 

Temperature: 

Operator: 

Humidity: 

ATTRIBUTE 

CATEGORY 

DESCRIPTION 

UNIT  OF 

MEASURE 

TEST  SCORE 

OPERATIONAL  QUESTIONS 

Set  up  Time  for  All 

Equipment 

Usability 

Time  to  set  up  all  equipment  (including  ancillary 
equipment)  for  use 

Seconds 

Programming  Time 

Usability 

Time  to  program  the  instrument  with  information 
about  sample  to  be  analyzed 

Seconds 

Training  Time  Required 

Training 

Time  to  become  trained  on  the  system 

Minutes 

Sample  Preparation  Time  in 
PPE  (MOPP  IV) 

Sample  Processing 

Time  required  from  receipt  of  sample  until  it  is 
ready  to  be  analyzed.  Operator  in  PPE. 

Minutes 

Sample  Run  Errors 

Usability 

Error  during  sample  analysis 

Yes/No 

Total  Analysis  Time 

Sample  Analysis 

The  duration  of  time  to  provide  final 
identification  of  target  from  raw  data. 

Minutes 

Diagnostic  Call  (Identification/Positive/Negative/Unsure) 

UNIT  OF 

TEST  SCORE 

MEASURE 

LOWEST 

POOR 

FAIR 

GOOD 

EXCELLENT 

Ease-of-Use 

Usability 

User's  Rating  of  ease  to  manipulate  buttons, 
process  samples,  etc. 

Rating 

Ease-of-Result  Viewing  and 
Interpretation  of  Data 

Usability 

User's  Rating 

Rating 

Supporting  Documentation 
(e.g.,  technical  manuals) 

Usability 

User's  Rating 

Rating 

Training  Simplicity 

Training 

User's  Rating 

Rating 

Safety 

Safety 

User's  Rating  of  safety 

Rating 

Cleaning/Maintenance 

Simplicity 

Logistical  Support 

User  Rating 

Rating 

Additional  Comments 
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